[PATCH 0/2] Last fixes for locales

[PATCH 0/2] Last fixes for locales

[Alexander Smirnov]

Hi all,

for today there are two locale problems in Isar:

1. During application's build the following is in log:

  /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) 

2. Log for isar-bootstrap contains messages like:

  perl: warning: Please check that your locale settings: 
      LANGUAGE = (unset), 
      LC_ALL = (unset), 
      LANG = "en_IN.ISO8859-1" 

This series does the following:
1. Moves locale settings from configscripts to isar-bootstrap.
2. Generate en_US.UTF-8 locale as the default one.

Now I haven't observed any locale-related messages.

Alex


Alexander Smirnov (2):
  isar-bootsrtap: Setup locale
  isar-bootstrap: Generate en_US.UTF-8 locale

 .../images/files/debian-configscript.sh            | 22 --------------------
 .../images/files/raspbian-configscript.sh          | 22 --------------------
 meta/recipes-core/isar-bootstrap/files/locale      |  4 ++++
 .../isar-bootstrap/files/locale.nopurge            | 10 +++++++++
 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 16 +++++++++++++--
 .../buildchroot/files/configscript.sh              | 24 +---------------------
 6 files changed, 29 insertions(+), 69 deletions(-)
 create mode 100644 meta/recipes-core/isar-bootstrap/files/locale
 create mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge

-- 
2.1.4

[PATCH 1/2] isar-bootsrtap: Setup locale

[Alexander Smirnov]

Currently locale is set in config scripts for buildchroot and images.
This is absolutely the same routine which is duplicated in all these
scripts. This patch moves locale initialization to isar-bootstrap, so
now locale is set once and works for all the futher root filesystems.

The task do_set_locale is added right after do_bootstrap to avoid warning
messages in isar-bootstrap logs after 'apt-get' calls.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 .../images/files/debian-configscript.sh            | 22 --------------------
 .../images/files/raspbian-configscript.sh          | 22 --------------------
 meta/recipes-core/isar-bootstrap/files/locale      |  4 ++++
 .../isar-bootstrap/files/locale.nopurge            | 10 +++++++++
 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 13 ++++++++++--
 .../buildchroot/files/configscript.sh              | 24 +---------------------
 6 files changed, 26 insertions(+), 69 deletions(-)
 create mode 100644 meta/recipes-core/isar-bootstrap/files/locale
 create mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge

diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/recipes-core/images/files/debian-configscript.sh
index 042b530..7ca0562 100755
--- a/meta-isar/recipes-core/images/files/debian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
@@ -10,28 +10,6 @@ readonly BAUDRATE_TTY="$2"
 readonly ROOTFS_DEV="$3"
 readonly ROOTFS_TYPE="$4"
 
-cat >> /etc/default/locale << EOF
-LANG=en_US.UTF-8
-LANGUAGE=en_US.UTF-8
-LC_ALL=C
-LC_CTYPE=C
-EOF
-
-## Configuration file for localepurge(8)
-cat > /etc/locale.nopurge << EOF
-
-# Remove localized man pages
-MANDELETE
-
-# Delete new locales which appear on the system without bothering you
-DONTBOTHERNEWLOCALE
-
-# Keep these locales after package installations via apt-get(8)
-en
-en_US
-en_US.UTF-8
-EOF
-
 debconf-set-selections <<END
 locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
 locales locales/default_environment_locale select en_US.UTF-8
diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
index ec05a6b..b240de9 100644
--- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
@@ -10,28 +10,6 @@ readonly BAUDRATE_TTY="$2"
 readonly ROOTFS_DEV="$3"
 readonly ROOTFS_TYPE="$4"
 
-cat >> /etc/default/locale << EOF
-LANG=en_US.UTF-8
-LANGUAGE=en_US.UTF-8
-LC_ALL=C
-LC_CTYPE=C
-EOF
-
-## Configuration file for localepurge(8)
-cat > /etc/locale.nopurge << EOF
-
-# Remove localized man pages
-MANDELETE
-
-# Delete new locales which appear on the system without bothering you
-DONTBOTHERNEWLOCALE
-
-# Keep these locales after package installations via apt-get(8)
-en
-en_US
-en_US.UTF-8
-EOF
-
 debconf-set-selections <<END
 locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
 locales locales/default_environment_locale select en_US.UTF-8
diff --git a/meta/recipes-core/isar-bootstrap/files/locale b/meta/recipes-core/isar-bootstrap/files/locale
new file mode 100644
index 0000000..5b3a663
--- /dev/null
+++ b/meta/recipes-core/isar-bootstrap/files/locale
@@ -0,0 +1,4 @@
+LANG=en_US.UTF-8
+LANGUAGE=en_US.UTF-8
+LC_ALL=C
+LC_CTYPE=C
diff --git a/meta/recipes-core/isar-bootstrap/files/locale.nopurge b/meta/recipes-core/isar-bootstrap/files/locale.nopurge
new file mode 100644
index 0000000..510d40c
--- /dev/null
+++ b/meta/recipes-core/isar-bootstrap/files/locale.nopurge
@@ -0,0 +1,10 @@
+# Remove localized man pages
+MANDELETE
+
+# Delete new locales which appear on the system without bothering you
+DONTBOTHERNEWLOCALE
+
+# Keep these locales after package installations via apt-get(8)
+en
+en_US
+en_US.UTF-8
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
index 35953b5..28ad395 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
@@ -12,7 +12,9 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260
 FILESPATH_prepend := "${THISDIR}/files:"
 SRC_URI = " \
     file://isar-apt.conf \
-    file://isar-apt-fallback.conf"
+    file://isar-apt-fallback.conf \
+    file://locale \
+    file://locale.nopurge"
 PV = "1.0"
 
 WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
@@ -183,6 +185,7 @@ do_bootstrap() {
     sudo -E "${DEBOOTSTRAP}" --verbose \
                              --variant=minbase \
                              --arch="${DISTRO_ARCH}" \
+                             --include=locales \
                              ${@get_distro_components_argument(d)} \
                              ${DEBOOTSTRAP_KEYRING} \
                              "${@get_distro_suite(d)}" \
@@ -191,6 +194,12 @@ do_bootstrap() {
 }
 addtask bootstrap before do_build after do_generate_keyring
 
+do_set_locale() {
+    sudo install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
+    sudo install -v -m644 "${WORKDIR}/locale.nopurge" "${ROOTFSDIR}/etc/locale.nopurge"
+}
+addtask set_locale after do_bootstrap
+
 def get_host_release():
     import platform
     rel = platform.release()
@@ -227,7 +236,7 @@ do_apt_update() {
     sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
                                       -o Debug::pkgProblemResolver=yes
 }
-addtask apt_update before do_build after do_apt_config_install
+addtask apt_update before do_build after do_apt_config_install do_set_locale
 
 do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh
index c2a599a..d591c2a 100644
--- a/meta/recipes-devtools/buildchroot/files/configscript.sh
+++ b/meta/recipes-devtools/buildchroot/files/configscript.sh
@@ -1,32 +1,10 @@
 #!/bin/sh
 #
 # This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
+# Copyright (C) 2015-2018 ilbers GmbH
 
 set -e
 
-cat >> /etc/default/locale << EOF
-LANG=en_US.UTF-8
-LANGUAGE=en_US.UTF-8
-LC_ALL=C
-LC_CTYPE=C
-EOF
-
-## Configuration file for localepurge(8)
-cat > /etc/locale.nopurge << EOF
-
-# Remove localized man pages
-MANDELETE
-
-# Delete new locales which appear on the system without bothering you
-DONTBOTHERNEWLOCALE
-
-# Keep these locales after package installations via apt-get(8)
-en
-en_US
-en_US.UTF-8
-EOF
-
 debconf-set-selections <<END
 locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
 locales locales/default_environment_locale select en_US.UTF-8
-- 
2.1.4

[PATCH 2/2] isar-bootstrap: Generate en_US.UTF-8 locale

[Alexander Smirnov]

Generate this locale by default for all the root filesystems used in Isar.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
index 28ad395..937ac9d 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
@@ -197,6 +197,9 @@ addtask bootstrap before do_build after do_generate_keyring
 do_set_locale() {
     sudo install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
     sudo install -v -m644 "${WORKDIR}/locale.nopurge" "${ROOTFSDIR}/etc/locale.nopurge"
+
+    sudo sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen"
+    sudo -E chroot "${ROOTFSDIR}" /usr/sbin/locale-gen
 }
 addtask set_locale after do_bootstrap
 
-- 
2.1.4

Re: [PATCH 1/2] isar-bootsrtap: Setup locale

[Claudius Heine]

Hi Alex,

On 2018-05-15 17:30, Alexander Smirnov wrote:
> Currently locale is set in config scripts for buildchroot and images.
> This is absolutely the same routine which is duplicated in all these
> scripts. This patch moves locale initialization to isar-bootstrap, so
> now locale is set once and works for all the futher root filesystems.
> 
> The task do_set_locale is added right after do_bootstrap to avoid warning
> messages in isar-bootstrap logs after 'apt-get' calls.
> 
> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> ---
>   .../images/files/debian-configscript.sh            | 22 --------------------
>   .../images/files/raspbian-configscript.sh          | 22 --------------------
>   meta/recipes-core/isar-bootstrap/files/locale      |  4 ++++
>   .../isar-bootstrap/files/locale.nopurge            | 10 +++++++++
>   meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 13 ++++++++++--
>   .../buildchroot/files/configscript.sh              | 24 +---------------------
>   6 files changed, 26 insertions(+), 69 deletions(-)
>   create mode 100644 meta/recipes-core/isar-bootstrap/files/locale
>   create mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge
> 
> diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/recipes-core/images/files/debian-configscript.sh
> index 042b530..7ca0562 100755
> --- a/meta-isar/recipes-core/images/files/debian-configscript.sh
> +++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
> @@ -10,28 +10,6 @@ readonly BAUDRATE_TTY="$2"
>   readonly ROOTFS_DEV="$3"
>   readonly ROOTFS_TYPE="$4"
>   
> -cat >> /etc/default/locale << EOF
> -LANG=en_US.UTF-8
> -LANGUAGE=en_US.UTF-8
> -LC_ALL=C
> -LC_CTYPE=C
> -EOF
> -
> -## Configuration file for localepurge(8)
> -cat > /etc/locale.nopurge << EOF
> -
> -# Remove localized man pages
> -MANDELETE
> -
> -# Delete new locales which appear on the system without bothering you
> -DONTBOTHERNEWLOCALE
> -
> -# Keep these locales after package installations via apt-get(8)
> -en
> -en_US
> -en_US.UTF-8
> -EOF
> -
>   debconf-set-selections <<END
>   locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
>   locales locales/default_environment_locale select en_US.UTF-8
> diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
> index ec05a6b..b240de9 100644
> --- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
> +++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
> @@ -10,28 +10,6 @@ readonly BAUDRATE_TTY="$2"
>   readonly ROOTFS_DEV="$3"
>   readonly ROOTFS_TYPE="$4"
>   
> -cat >> /etc/default/locale << EOF
> -LANG=en_US.UTF-8
> -LANGUAGE=en_US.UTF-8
> -LC_ALL=C
> -LC_CTYPE=C
> -EOF
> -
> -## Configuration file for localepurge(8)
> -cat > /etc/locale.nopurge << EOF
> -
> -# Remove localized man pages
> -MANDELETE
> -
> -# Delete new locales which appear on the system without bothering you
> -DONTBOTHERNEWLOCALE
> -
> -# Keep these locales after package installations via apt-get(8)
> -en
> -en_US
> -en_US.UTF-8
> -EOF
> -
>   debconf-set-selections <<END
>   locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
>   locales locales/default_environment_locale select en_US.UTF-8
> diff --git a/meta/recipes-core/isar-bootstrap/files/locale b/meta/recipes-core/isar-bootstrap/files/locale
> new file mode 100644
> index 0000000..5b3a663
> --- /dev/null
> +++ b/meta/recipes-core/isar-bootstrap/files/locale
> @@ -0,0 +1,4 @@
> +LANG=en_US.UTF-8
> +LANGUAGE=en_US.UTF-8
> +LC_ALL=C
> +LC_CTYPE=C
> diff --git a/meta/recipes-core/isar-bootstrap/files/locale.nopurge b/meta/recipes-core/isar-bootstrap/files/locale.nopurge
> new file mode 100644
> index 0000000..510d40c
> --- /dev/null
> +++ b/meta/recipes-core/isar-bootstrap/files/locale.nopurge
> @@ -0,0 +1,10 @@
> +# Remove localized man pages
> +MANDELETE
> +
> +# Delete new locales which appear on the system without bothering you
> +DONTBOTHERNEWLOCALE
> +
> +# Keep these locales after package installations via apt-get(8)
> +en
> +en_US
> +en_US.UTF-8

I don't think that putting 'locale.nopurge' in every image is the right 
solution. That means that every image is configured for localepurge.

I'm working on a solution to allow configuration packages, that are 
installed and then purged from the image:

https://github.com/cmhe/isar/commit/624ee1278657689ad5ef493b345905308a340039

https://github.com/cmhe/isar/commit/765b6b09e32230cd6d1929fd9f54455cb4ff65fc

This is useful for one-shot configuration stuff like localepurge and 
maybe even the complete locale configuration.

But I do see the reason for setting *some* locale variables in the 
isar-bootstrap, that can be easily overwritten later on just to provide 
a baseline configuration that avoids those warnings.

Claudius


> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> index 35953b5..28ad395 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> @@ -12,7 +12,9 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260
>   FILESPATH_prepend := "${THISDIR}/files:"
>   SRC_URI = " \
>       file://isar-apt.conf \
> -    file://isar-apt-fallback.conf"
> +    file://isar-apt-fallback.conf \
> +    file://locale \
> +    file://locale.nopurge"
>   PV = "1.0"
>   
>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
> @@ -183,6 +185,7 @@ do_bootstrap() {
>       sudo -E "${DEBOOTSTRAP}" --verbose \
>                                --variant=minbase \
>                                --arch="${DISTRO_ARCH}" \
> +                             --include=locales \
>                                ${@get_distro_components_argument(d)} \
>                                ${DEBOOTSTRAP_KEYRING} \
>                                "${@get_distro_suite(d)}" \
> @@ -191,6 +194,12 @@ do_bootstrap() {
>   }
>   addtask bootstrap before do_build after do_generate_keyring
>   
> +do_set_locale() {
> +    sudo install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
> +    sudo install -v -m644 "${WORKDIR}/locale.nopurge" "${ROOTFSDIR}/etc/locale.nopurge"
> +}
> +addtask set_locale after do_bootstrap
> +
>   def get_host_release():
>       import platform
>       rel = platform.release()
> @@ -227,7 +236,7 @@ do_apt_update() {
>       sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
>                                         -o Debug::pkgProblemResolver=yes
>   }
> -addtask apt_update before do_build after do_apt_config_install
> +addtask apt_update before do_build after do_apt_config_install do_set_locale
>   
>   do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>   do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
> diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh
> index c2a599a..d591c2a 100644
> --- a/meta/recipes-devtools/buildchroot/files/configscript.sh
> +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh
> @@ -1,32 +1,10 @@
>   #!/bin/sh
>   #
>   # This software is a part of ISAR.
> -# Copyright (C) 2015-2017 ilbers GmbH
> +# Copyright (C) 2015-2018 ilbers GmbH
>   
>   set -e
>   
> -cat >> /etc/default/locale << EOF
> -LANG=en_US.UTF-8
> -LANGUAGE=en_US.UTF-8
> -LC_ALL=C
> -LC_CTYPE=C
> -EOF
> -
> -## Configuration file for localepurge(8)
> -cat > /etc/locale.nopurge << EOF
> -
> -# Remove localized man pages
> -MANDELETE
> -
> -# Delete new locales which appear on the system without bothering you
> -DONTBOTHERNEWLOCALE
> -
> -# Keep these locales after package installations via apt-get(8)
> -en
> -en_US
> -en_US.UTF-8
> -EOF
> -
>   debconf-set-selections <<END
>   locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
>   locales locales/default_environment_locale select en_US.UTF-8
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH 0/2] Last fixes for locales

[Claudius Heine]

On 2018-05-15 17:30, Alexander Smirnov wrote:
> Hi all,
> 
> for today there are two locale problems in Isar:
> 
> 1. During application's build the following is in log:
> 
>    /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> 
> 2. Log for isar-bootstrap contains messages like:
> 
>    perl: warning: Please check that your locale settings:
>        LANGUAGE = (unset),
>        LC_ALL = (unset),
>        LANG = "en_IN.ISO8859-1"
> 
> This series does the following:
> 1. Moves locale settings from configscripts to isar-bootstrap.
> 2. Generate en_US.UTF-8 locale as the default one.
> 
> Now I haven't observed any locale-related messages.

I just discovered: merged


-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de