[PATCH v4 0/3] One-shot configuration packages

[PATCH v4 0/3] One-shot configuration packages

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Hi,

I implemented Hennings suggestion to purge the one-shot packages in the
reverse order of the installation.

Claudius

Changes from v3:
 - Purge one-shot packages in reverse order

Changes from v2:
 - Install one-shot package with a loop
 - Rename '-y' apt parameter to '--yes'
 - Move isar-cfg-localepurge files from /usr/local/etc to /usr/lib/

Changes from v1:
 - rebased to current next
 - removed locale.nopurge setup in isar-bootstrap

Claudius Heine (3):
  isar-bootstrap|configscript: implement one shot config option
  meta/isar-cfg-localepurge: added locale purge recipe
  meta-isar/isar-image-base: switch use isar-cfg-localepurge

 .../images/files/debian-configscript.sh       |  9 ------
 .../images/files/raspbian-configscript.sh     |  8 -----
 .../recipes-core/images/isar-image-base.bb    |  6 ++--
 meta/classes/isar-bootstrap-helper.bbclass    | 32 ++++++++++++++++---
 .../isar-bootstrap/files/locale.nopurge       | 10 ------
 .../isar-bootstrap/isar-bootstrap.bb          |  4 +--
 .../isar-cfg-localepurge/files/locale.debconf | 12 +++++++
 .../isar-cfg-localepurge/files/locale.gen     |  1 +
 .../isar-cfg-localepurge/files/postinst       | 30 +++++++++++++++++
 .../isar-cfg-localepurge.bb                   | 19 +++++++++++
 10 files changed, 94 insertions(+), 37 deletions(-)
 delete mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.debconf
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.gen
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/postinst
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb

-- 
2.17.0

[PATCH v4 1/3] isar-bootstrap|configscript: implement one shot config option

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Some configuration tasks on the image require certain other packages,
but those packages might not be required on the final image.

One way to solve this is use a special package that is installed to the
image and then removed again. When installing it makes certain that all
required packages are installed to the image as well. Then the image
can be configured in the postinst hook. When this package is removed and
all unused dependencies are removed as well, no not required packages
used by this package should remain on the image.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 .../images/files/debian-configscript.sh       |  1 -
 .../images/files/raspbian-configscript.sh     |  1 -
 meta/classes/isar-bootstrap-helper.bbclass    | 32 ++++++++++++++++---
 3 files changed, 28 insertions(+), 6 deletions(-)

diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/recipes-core/images/files/debian-configscript.sh
index 7ca0562..9b3ff30 100755
--- a/meta-isar/recipes-core/images/files/debian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
@@ -40,4 +40,3 @@ fi
 
 # Purge unused locale and installed packages' .deb files
 localepurge
-apt-get clean
diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
index b240de9..448dea5 100644
--- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
@@ -40,4 +40,3 @@ kernel=$KERNEL_IMAGE
 EOF
 
 localepurge
-apt-get clean
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 4284d02..b87c94e 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -5,15 +5,24 @@
 #
 # SPDX-License-Identifier: MIT
 
+IMAGE_CFG_PACKAGE ??= ""
+
+def reverse_bb_array(d, varname):
+    array = d.getVar(varname, True)
+    if array is None:
+        return None
+    array = reversed(array.split())
+    return " ".join(i for i in array)
+
 setup_root_file_system() {
     ROOTFSDIR="$1"
-    CLEANHOSTLEAK="$2"
+    CLEAN="$2"
     shift
     shift
     PACKAGES="$@"
     APT_ARGS="install --yes --allow-unauthenticated \
               -o Debug::pkgProblemResolver=yes"
-    CLEANHOSTLEAK_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
+    CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
 
     sudo cp -Trpfx \
         "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}/" \
@@ -41,8 +50,23 @@ setup_root_file_system() {
         -o Dir::Etc::sourceparts="-" \
         -o APT::Get::List-Cleanup="0"
     sudo -E chroot "$ROOTFSDIR" \
-        /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES
-    [ "clean" = ${CLEANHOSTLEAK} ] && sudo rm -f ${CLEANHOSTLEAK_FILES}
+        /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
+            ${IMAGE_CFG_PACKAGE}
+    [ "clean" = ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
     sudo -E chroot "$ROOTFSDIR" \
         /usr/bin/apt-get ${APT_ARGS} $PACKAGES
+    for pkg in ${IMAGE_CFG_PACKAGE}; do
+        sudo -E chroot "$ROOTFSDIR" \
+            /usr/bin/apt-get ${APT_ARGS} $pkg
+    done
+    for pkg in ${@reverse_bb_array(d, "IMAGE_CFG_PACKAGE") or ""}; do
+        sudo -E chroot "$ROOTFSDIR" \
+            /usr/bin/apt-get purge --yes $pkg
+    done
+    if [ "clean" = ${CLEAN} ]; then
+        sudo -E chroot "$ROOTFSDIR" \
+            /usr/bin/apt-get autoremove --purge --yes
+        sudo -E chroot "$ROOTFSDIR" \
+            /usr/bin/apt-get clean
+    fi
 }
-- 
2.17.0

[PATCH v4 2/3] meta/isar-cfg-localepurge: added locale purge recipe

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

The package 'isar-cfg-localepurge' is used to set the locales of the
root file system and purge all not required locales.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 .../isar-cfg-localepurge/files/locale.debconf | 12 ++++++++
 .../isar-cfg-localepurge/files/locale.gen     |  1 +
 .../isar-cfg-localepurge/files/postinst       | 30 +++++++++++++++++++
 .../isar-cfg-localepurge.bb                   | 19 ++++++++++++
 4 files changed, 62 insertions(+)
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.debconf
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.gen
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/postinst
 create mode 100644 meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb

diff --git a/meta/recipes-support/isar-cfg-localepurge/files/locale.debconf b/meta/recipes-support/isar-cfg-localepurge/files/locale.debconf
new file mode 100644
index 0000000..169c071
--- /dev/null
+++ b/meta/recipes-support/isar-cfg-localepurge/files/locale.debconf
@@ -0,0 +1,12 @@
+locales	locales/locales_to_be_generated	multiselect	en_US.UTF-8 UTF-8
+locales	locales/default_environment_locale	select	en_US.UTF-8
+
+localepurge	localepurge/dontbothernew	boolean	true
+localepurge	localepurge/nopurge	multiselect	en, en_US, en_US.UTF-8
+localepurge	localepurge/use-dpkg-feature	boolean	false
+localepurge	localepurge/verbose	boolean	false
+localepurge	localepurge/showfreedspace	boolean	false
+localepurge	localepurge/none_selected	boolean	false
+localepurge	localepurge/mandelete	boolean	true
+localepurge	localepurge/quickndirtycalc	boolean	false
+localepurge	localepurge/remove_no	note
diff --git a/meta/recipes-support/isar-cfg-localepurge/files/locale.gen b/meta/recipes-support/isar-cfg-localepurge/files/locale.gen
new file mode 100644
index 0000000..a66d814
--- /dev/null
+++ b/meta/recipes-support/isar-cfg-localepurge/files/locale.gen
@@ -0,0 +1 @@
+en_US.UTF-8 UTF-8
diff --git a/meta/recipes-support/isar-cfg-localepurge/files/postinst b/meta/recipes-support/isar-cfg-localepurge/files/postinst
new file mode 100644
index 0000000..092f12e
--- /dev/null
+++ b/meta/recipes-support/isar-cfg-localepurge/files/postinst
@@ -0,0 +1,30 @@
+#!/bin/sh
+set -e
+
+cat /usr/lib/isar-cfg-localepurge/locale.gen \
+	>> /etc/locale.gen
+
+debconf-set-selections /usr/lib/isar-cfg-localepurge/locale.debconf
+
+# locale.nopurge needs to be removed before localepurge is reconfigured.
+# Otherwise it would set the debconf to the values from the locale.nopurge
+# file again.
+rm -rf /etc/locale.nopurge
+dpkg-reconfigure -f noninteractive locales localepurge
+
+# When /etc/locale.nopurge was removed before dpkg-reconfigure, it writes
+# the new configuration with the '.ucf-dist' postfix
+mv /etc/locale.nopurge.ucf-dist /etc/locale.nopurge
+
+# Now reconfigure it localpurge again, because otherwise it would complain:
+#     Some new locales have appeared on your system:
+#
+#     bal be@latin en@boldquot en@quot sr@latin
+#
+#     They will not be touched until you reconfigure localepurge
+#     with the following command:
+#
+#         dpkg-reconfigure localepurge
+dpkg-reconfigure -f noninteractive localepurge
+
+localepurge
diff --git a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
new file mode 100644
index 0000000..d2cf1f3
--- /dev/null
+++ b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
@@ -0,0 +1,19 @@
+# This software is a part of ISAR.
+
+DESCRIPTION = "Isar configuration package for locale and localepurge"
+MAINTAINER = "isar-users <isar-users@googlegroups.com>"
+DEBIAN_DEPENDS = "localepurge"
+
+SRC_URI = "file://locale.debconf \
+	   file://locale.gen \
+	   file://postinst"
+
+inherit dpkg-raw
+
+do_install() {
+	install -v -d ${D}/usr/lib/${PN}
+	install -v -m 644 ${WORKDIR}/locale.debconf \
+                          ${D}/usr/lib/${PN}/locale.debconf
+	install -v -m 644 ${WORKDIR}/locale.gen \
+                          ${D}/usr/lib/${PN}/locale.gen
+}
-- 
2.17.0

[PATCH v4 3/3] meta-isar/isar-image-base: switch use isar-cfg-localepurge

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

This patch removes the locale setting from the *-configscripts and
instead uses the isar-cfg-localepurge configuration package.

The advantange is that the package 'localepurge' is no longer installed
on every image and locale settings is done in a central place.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 .../recipes-core/images/files/debian-configscript.sh   |  8 --------
 .../recipes-core/images/files/raspbian-configscript.sh |  7 -------
 meta-isar/recipes-core/images/isar-image-base.bb       |  6 +++---
 meta/recipes-core/isar-bootstrap/files/locale.nopurge  | 10 ----------
 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb     |  4 +---
 5 files changed, 4 insertions(+), 31 deletions(-)
 delete mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge

diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/recipes-core/images/files/debian-configscript.sh
index 9b3ff30..136918c 100755
--- a/meta-isar/recipes-core/images/files/debian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
@@ -10,11 +10,6 @@ readonly BAUDRATE_TTY="$2"
 readonly ROOTFS_DEV="$3"
 readonly ROOTFS_TYPE="$4"
 
-debconf-set-selections <<END
-locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
-locales locales/default_environment_locale select en_US.UTF-8
-END
-
 cat > /etc/fstab << EOF
 # Begin /etc/fstab
 /dev/$ROOTFS_DEV	/		$ROOTFS_TYPE		defaults		1	1
@@ -37,6 +32,3 @@ if [ -f /etc/inittab ]; then
     echo "T0:23:respawn:/sbin/getty -L $MACHINE_SERIAL $BAUDRATE_TTY vt100" \
         >> /etc/inittab
 fi
-
-# Purge unused locale and installed packages' .deb files
-localepurge
diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
index 448dea5..04ea571 100644
--- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
@@ -10,11 +10,6 @@ readonly BAUDRATE_TTY="$2"
 readonly ROOTFS_DEV="$3"
 readonly ROOTFS_TYPE="$4"
 
-debconf-set-selections <<END
-locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
-locales locales/default_environment_locale select en_US.UTF-8
-END
-
 cat > /etc/fstab << EOF
 # Begin /etc/fstab
 /dev/$ROOTFS_DEV	/		$ROOTFS_TYPE		defaults		1	1
@@ -38,5 +33,3 @@ KERNEL_IMAGE=`ls /boot | grep vmlinuz`
 cat > /boot/config.txt << EOF
 kernel=$KERNEL_IMAGE
 EOF
-
-localepurge
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 1239f72..0d46d74 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -16,11 +16,11 @@ PV = "1.0"
 inherit image
 inherit isar-bootstrap-helper
 
-DEPENDS += "${IMAGE_INSTALL}"
+DEPENDS += "${IMAGE_INSTALL} ${IMAGE_CFG_PACKAGE}"
 
 IMAGE_PREINSTALL += "apt \
-                     dbus \
-                     localepurge"
+                     dbus"
+IMAGE_CFG_PACKAGE += "isar-cfg-localepurge"
 
 WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 
diff --git a/meta/recipes-core/isar-bootstrap/files/locale.nopurge b/meta/recipes-core/isar-bootstrap/files/locale.nopurge
deleted file mode 100644
index 510d40c..0000000
--- a/meta/recipes-core/isar-bootstrap/files/locale.nopurge
+++ /dev/null
@@ -1,10 +0,0 @@
-# Remove localized man pages
-MANDELETE
-
-# Delete new locales which appear on the system without bothering you
-DONTBOTHERNEWLOCALE
-
-# Keep these locales after package installations via apt-get(8)
-en
-en_US
-en_US.UTF-8
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
index f894821..bb3992b 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
@@ -13,8 +13,7 @@ FILESPATH_prepend := "${THISDIR}/files:"
 SRC_URI = " \
     file://isar-apt.conf \
     file://isar-apt-fallback.conf \
-    file://locale \
-    file://locale.nopurge"
+    file://locale"
 PV = "1.0"
 
 WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
@@ -196,7 +195,6 @@ addtask bootstrap before do_build after do_generate_keyring
 
 do_set_locale() {
     sudo install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
-    sudo install -v -m644 "${WORKDIR}/locale.nopurge" "${ROOTFSDIR}/etc/locale.nopurge"
 
     sudo sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen"
     sudo -E chroot "${ROOTFSDIR}" /usr/sbin/locale-gen
-- 
2.17.0

Re: [PATCH v4 1/3] isar-bootstrap|configscript: implement one shot config option

[Maxim Yu. Osipov]

Hi Claudius,

On 05/28/2018 12:56 PM, claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
> 
> Some configuration tasks on the image require certain other packages,
> but those packages might not be required on the final image.
> 
> One way to solve this is use a special package that is installed to the
> image and then removed again. When installing it makes certain that all
> required packages are installed to the image as well. Then the image
> can be configured in the postinst hook. When this package is removed and
> all unused dependencies are removed as well, no not required packages
> used by this package should remain on the image.
> 
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
>   .../images/files/debian-configscript.sh       |  1 -
>   .../images/files/raspbian-configscript.sh     |  1 -
>   meta/classes/isar-bootstrap-helper.bbclass    | 32 ++++++++++++++++---
>   3 files changed, 28 insertions(+), 6 deletions(-)
> 
> diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/recipes-core/images/files/debian-configscript.sh
> index 7ca0562..9b3ff30 100755
> --- a/meta-isar/recipes-core/images/files/debian-configscript.sh
> +++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
> @@ -40,4 +40,3 @@ fi
>   
>   # Purge unused locale and installed packages' .deb files
>   localepurge
> -apt-get clean
> diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
> index b240de9..448dea5 100644
> --- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
> +++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
> @@ -40,4 +40,3 @@ kernel=$KERNEL_IMAGE
>   EOF
>   
>   localepurge
> -apt-get clean
> diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
> index 4284d02..b87c94e 100644
> --- a/meta/classes/isar-bootstrap-helper.bbclass
> +++ b/meta/classes/isar-bootstrap-helper.bbclass
> @@ -5,15 +5,24 @@
>   #
>   # SPDX-License-Identifier: MIT
>   
> +IMAGE_CFG_PACKAGE ??= ""

Are you going to rename this variable to IMAGE_CFG_PACKAGES
and add a comment it in a separate patch?

Kind regards,
Maxim.

> +
> +def reverse_bb_array(d, varname):
> +    array = d.getVar(varname, True)
> +    if array is None:
> +        return None
> +    array = reversed(array.split())
> +    return " ".join(i for i in array)
> +
>   setup_root_file_system() {
>       ROOTFSDIR="$1"
> -    CLEANHOSTLEAK="$2"
> +    CLEAN="$2"
>       shift
>       shift
>       PACKAGES="$@"
>       APT_ARGS="install --yes --allow-unauthenticated \
>                 -o Debug::pkgProblemResolver=yes"
> -    CLEANHOSTLEAK_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
> +    CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
>   
>       sudo cp -Trpfx \
>           "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}/" \
> @@ -41,8 +50,23 @@ setup_root_file_system() {
>           -o Dir::Etc::sourceparts="-" \
>           -o APT::Get::List-Cleanup="0"
>       sudo -E chroot "$ROOTFSDIR" \
> -        /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES
> -    [ "clean" = ${CLEANHOSTLEAK} ] && sudo rm -f ${CLEANHOSTLEAK_FILES}
> +        /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
> +            ${IMAGE_CFG_PACKAGE}
> +    [ "clean" = ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
>       sudo -E chroot "$ROOTFSDIR" \
>           /usr/bin/apt-get ${APT_ARGS} $PACKAGES
> +    for pkg in ${IMAGE_CFG_PACKAGE}; do
> +        sudo -E chroot "$ROOTFSDIR" \
> +            /usr/bin/apt-get ${APT_ARGS} $pkg
> +    done
> +    for pkg in ${@reverse_bb_array(d, "IMAGE_CFG_PACKAGE") or ""}; do
> +        sudo -E chroot "$ROOTFSDIR" \
> +            /usr/bin/apt-get purge --yes $pkg
> +    done
> +    if [ "clean" = ${CLEAN} ]; then
> +        sudo -E chroot "$ROOTFSDIR" \
> +            /usr/bin/apt-get autoremove --purge --yes
> +        sudo -E chroot "$ROOTFSDIR" \
> +            /usr/bin/apt-get clean
> +    fi
>   }
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v4 0/3] One-shot configuration packages

[Baurzhan Ismagulov]

Hello Claudius,

this is a useful feature that is also user-visible. Would you like to provide a
patch to doc/user_manual.md? I think we should provide a recommendation for the
user how to do customizations. I'd be especially interested in your vision
where to provide what, e.g.: One or many customization packages? If many, how
to split customizations between packages? Or just describe the mechanism and
leave the splitting decision to the user? In the latter case, would that mean
that in Isar, we'll by default provide only one package for all customizations?

With kind regards,
Baurzhan.


On Mon, May 28, 2018 at 12:56:40PM +0200, claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
> 
> Hi,
> 
> I implemented Hennings suggestion to purge the one-shot packages in the
> reverse order of the installation.
> 
> Claudius
> 
> Changes from v3:
>  - Purge one-shot packages in reverse order
> 
> Changes from v2:
>  - Install one-shot package with a loop
>  - Rename '-y' apt parameter to '--yes'
>  - Move isar-cfg-localepurge files from /usr/local/etc to /usr/lib/
> 
> Changes from v1:
>  - rebased to current next
>  - removed locale.nopurge setup in isar-bootstrap
> 
> Claudius Heine (3):
>   isar-bootstrap|configscript: implement one shot config option
>   meta/isar-cfg-localepurge: added locale purge recipe
>   meta-isar/isar-image-base: switch use isar-cfg-localepurge
> 
>  .../images/files/debian-configscript.sh       |  9 ------
>  .../images/files/raspbian-configscript.sh     |  8 -----
>  .../recipes-core/images/isar-image-base.bb    |  6 ++--
>  meta/classes/isar-bootstrap-helper.bbclass    | 32 ++++++++++++++++---
>  .../isar-bootstrap/files/locale.nopurge       | 10 ------
>  .../isar-bootstrap/isar-bootstrap.bb          |  4 +--
>  .../isar-cfg-localepurge/files/locale.debconf | 12 +++++++
>  .../isar-cfg-localepurge/files/locale.gen     |  1 +
>  .../isar-cfg-localepurge/files/postinst       | 30 +++++++++++++++++
>  .../isar-cfg-localepurge.bb                   | 19 +++++++++++
>  10 files changed, 94 insertions(+), 37 deletions(-)
>  delete mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge
>  create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.debconf
>  create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.gen
>  create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/postinst
>  create mode 100644 meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
> 
> -- 
> 2.17.0

Re: [PATCH v4 1/3] isar-bootstrap|configscript: implement one shot config option

[Claudius Heine]

Hi Maxim,

On 2018-05-29 09:58, Maxim Yu. Osipov wrote:
> Hi Claudius,
> 
> On 05/28/2018 12:56 PM, claudius.heine.ext@siemens.com wrote:
>> From: Claudius Heine <ch@denx.de>
>>
>> Some configuration tasks on the image require certain other packages,
>> but those packages might not be required on the final image.
>>
>> One way to solve this is use a special package that is installed to the
>> image and then removed again. When installing it makes certain that all
>> required packages are installed to the image as well. Then the image
>> can be configured in the postinst hook. When this package is removed and
>> all unused dependencies are removed as well, no not required packages
>> used by this package should remain on the image.
>>
>> Signed-off-by: Claudius Heine <ch@denx.de>
>> ---
>>   .../images/files/debian-configscript.sh       |  1 -
>>   .../images/files/raspbian-configscript.sh     |  1 -
>>   meta/classes/isar-bootstrap-helper.bbclass    | 32 ++++++++++++++++---
>>   3 files changed, 28 insertions(+), 6 deletions(-)
>>
>> diff --git 
>> a/meta-isar/recipes-core/images/files/debian-configscript.sh 
>> b/meta-isar/recipes-core/images/files/debian-configscript.sh
>> index 7ca0562..9b3ff30 100755
>> --- a/meta-isar/recipes-core/images/files/debian-configscript.sh
>> +++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
>> @@ -40,4 +40,3 @@ fi
>>   # Purge unused locale and installed packages' .deb files
>>   localepurge
>> -apt-get clean
>> diff --git 
>> a/meta-isar/recipes-core/images/files/raspbian-configscript.sh 
>> b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
>> index b240de9..448dea5 100644
>> --- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
>> +++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
>> @@ -40,4 +40,3 @@ kernel=$KERNEL_IMAGE
>>   EOF
>>   localepurge
>> -apt-get clean
>> diff --git a/meta/classes/isar-bootstrap-helper.bbclass 
>> b/meta/classes/isar-bootstrap-helper.bbclass
>> index 4284d02..b87c94e 100644
>> --- a/meta/classes/isar-bootstrap-helper.bbclass
>> +++ b/meta/classes/isar-bootstrap-helper.bbclass
>> @@ -5,15 +5,24 @@
>>   #
>>   # SPDX-License-Identifier: MIT
>> +IMAGE_CFG_PACKAGE ??= ""
> 
> Are you going to rename this variable to IMAGE_CFG_PACKAGES
> and add a comment it in a separate patch?

Oh, sorry. Forgot about that. I will prepare v5.

Claudius

> 
> Kind regards,
> Maxim.
> 
>> +
>> +def reverse_bb_array(d, varname):
>> +    array = d.getVar(varname, True)
>> +    if array is None:
>> +        return None
>> +    array = reversed(array.split())
>> +    return " ".join(i for i in array)
>> +
>>   setup_root_file_system() {
>>       ROOTFSDIR="$1"
>> -    CLEANHOSTLEAK="$2"
>> +    CLEAN="$2"
>>       shift
>>       shift
>>       PACKAGES="$@"
>>       APT_ARGS="install --yes --allow-unauthenticated \
>>                 -o Debug::pkgProblemResolver=yes"
>> -    CLEANHOSTLEAK_FILES="${ROOTFSDIR}/etc/hostname 
>> ${ROOTFSDIR}/etc/resolv.conf"
>> +    CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
>>       sudo cp -Trpfx \
>>           
>> "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}/" \
>> @@ -41,8 +50,23 @@ setup_root_file_system() {
>>           -o Dir::Etc::sourceparts="-" \
>>           -o APT::Get::List-Cleanup="0"
>>       sudo -E chroot "$ROOTFSDIR" \
>> -        /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES
>> -    [ "clean" = ${CLEANHOSTLEAK} ] && sudo rm -f ${CLEANHOSTLEAK_FILES}
>> +        /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
>> +            ${IMAGE_CFG_PACKAGE}
>> +    [ "clean" = ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
>>       sudo -E chroot "$ROOTFSDIR" \
>>           /usr/bin/apt-get ${APT_ARGS} $PACKAGES
>> +    for pkg in ${IMAGE_CFG_PACKAGE}; do
>> +        sudo -E chroot "$ROOTFSDIR" \
>> +            /usr/bin/apt-get ${APT_ARGS} $pkg
>> +    done
>> +    for pkg in ${@reverse_bb_array(d, "IMAGE_CFG_PACKAGE") or ""}; do
>> +        sudo -E chroot "$ROOTFSDIR" \
>> +            /usr/bin/apt-get purge --yes $pkg
>> +    done
>> +    if [ "clean" = ${CLEAN} ]; then
>> +        sudo -E chroot "$ROOTFSDIR" \
>> +            /usr/bin/apt-get autoremove --purge --yes
>> +        sudo -E chroot "$ROOTFSDIR" \
>> +            /usr/bin/apt-get clean
>> +    fi
>>   }
>>
> 
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH v4 0/3] One-shot configuration packages

[Claudius Heine]

Hi Baurzhan,

On 2018-05-29 10:00, Baurzhan Ismagulov wrote:
> Hello Claudius,
> 
> this is a useful feature that is also user-visible. Would you like to provide a
> patch to doc/user_manual.md?

I will do that and send a v5. But I answer your questions here as well. 
For me its easier to explain something in a dialog that just writing 
docs to someone unknown. :)

> I think we should provide a recommendation for the
> user how to do customizations. I'd be especially interested in your vision
> where to provide what, e.g.: One or many customization packages? 

Currently I cannot think of anything else that could be done with those 
one-shot packages apart from localepurge. Is generally only useful if 
someone needs a tool to customize the root file system that shouldn't be 
part of the image later. Normal customization packages that just change 
or add some configuration files should not be one-shot packages, they 
don't require special tools and therefor its not use purging them to 
remove their dependencies.

I don't even know if it makes really sense to allow multiple packages as 
one-shot packages, but I did it because it might be useful in some 
strange corner cases for someone and wasn't a big effort to do.

> If many, how
> to split customizations between packages?

All the customization, apart from locale configuration should be in 
normal packages, however the user choose to do this. locale 
configuration should be done with overwriting isar-cfg-localepurge 
package or its files.

Maybe the name of that package is choosen poorly. I don't know.

> Or just describe the mechanism and leave the splitting decision to the user?

Yes. That would be useful. The user might want to mix and match 
different configuration fragments contained in multiple configuration 
packages.

> In the latter case, would that mean
> that in Isar, we'll by default provide only one package for all customizations?

I would think so. Maybe move the customization to a isar-cfg-example 
package?

Claudius

> 
> With kind regards,
> Baurzhan.
> 
> 
> On Mon, May 28, 2018 at 12:56:40PM +0200, claudius.heine.ext@siemens.com wrote:
>> From: Claudius Heine <ch@denx.de>
>>
>> Hi,
>>
>> I implemented Hennings suggestion to purge the one-shot packages in the
>> reverse order of the installation.
>>
>> Claudius
>>
>> Changes from v3:
>>   - Purge one-shot packages in reverse order
>>
>> Changes from v2:
>>   - Install one-shot package with a loop
>>   - Rename '-y' apt parameter to '--yes'
>>   - Move isar-cfg-localepurge files from /usr/local/etc to /usr/lib/
>>
>> Changes from v1:
>>   - rebased to current next
>>   - removed locale.nopurge setup in isar-bootstrap
>>
>> Claudius Heine (3):
>>    isar-bootstrap|configscript: implement one shot config option
>>    meta/isar-cfg-localepurge: added locale purge recipe
>>    meta-isar/isar-image-base: switch use isar-cfg-localepurge
>>
>>   .../images/files/debian-configscript.sh       |  9 ------
>>   .../images/files/raspbian-configscript.sh     |  8 -----
>>   .../recipes-core/images/isar-image-base.bb    |  6 ++--
>>   meta/classes/isar-bootstrap-helper.bbclass    | 32 ++++++++++++++++---
>>   .../isar-bootstrap/files/locale.nopurge       | 10 ------
>>   .../isar-bootstrap/isar-bootstrap.bb          |  4 +--
>>   .../isar-cfg-localepurge/files/locale.debconf | 12 +++++++
>>   .../isar-cfg-localepurge/files/locale.gen     |  1 +
>>   .../isar-cfg-localepurge/files/postinst       | 30 +++++++++++++++++
>>   .../isar-cfg-localepurge.bb                   | 19 +++++++++++
>>   10 files changed, 94 insertions(+), 37 deletions(-)
>>   delete mode 100644 meta/recipes-core/isar-bootstrap/files/locale.nopurge
>>   create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.debconf
>>   create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/locale.gen
>>   create mode 100644 meta/recipes-support/isar-cfg-localepurge/files/postinst
>>   create mode 100644 meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
>>
>> -- 
>> 2.17.0
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH v4 1/3] isar-bootstrap|configscript: implement one shot config option

[Claudius Heine]

On 2018-05-29 10:20, [ext] Claudius Heine wrote:
>>>   # SPDX-License-Identifier: MIT
>>> +IMAGE_CFG_PACKAGE ??= ""
>>
>> Are you going to rename this variable to IMAGE_CFG_PACKAGES
>> and add a comment it in a separate patch?
> 
> Oh, sorry. Forgot about that. I will prepare v5.

I have thought on this and I think that 'IMAGE_CFG_PACKAGES' is a bad 
variable name, since it implies to be the variable to put all 
configuration packages into.
How about 'IMAGE_ONESHOT_PACKAGES', 'IMAGE_VOLATILE_PACKAGES' or 
'IMAGE_TEMPORARY_PACKAGES'?

Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH v4 1/3] isar-bootstrap|configscript: implement one shot config option

[Maxim Yu. Osipov]

On 05/29/2018 12:56 PM, Claudius Heine wrote:
> On 2018-05-29 10:20, [ext] Claudius Heine wrote:
>>>>   # SPDX-License-Identifier: MIT
>>>> +IMAGE_CFG_PACKAGE ??= ""
>>>
>>> Are you going to rename this variable to IMAGE_CFG_PACKAGES
>>> and add a comment it in a separate patch?
>>
>> Oh, sorry. Forgot about that. I will prepare v5.
> 
> I have thought on this and I think that 'IMAGE_CFG_PACKAGES' is a bad 
> variable name, since it implies to be the variable to put all 
> configuration packages into.
> How about 'IMAGE_ONESHOT_PACKAGES', 'IMAGE_VOLATILE_PACKAGES' or 
> 'IMAGE_TEMPORARY_PACKAGES'?

The purpose of these packages is to do image post configuration (or 
tweaking)...
May be IMAGE_POSTCONFIG_PACKAGES or IMAGE_TWEAK_PACKAGES?

Maxim.


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v4 1/3] isar-bootstrap|configscript: implement one shot config option

[Claudius Heine]

Hi Maxim,

On 2018-05-29 13:18, Maxim Yu. Osipov wrote:
> On 05/29/2018 12:56 PM, Claudius Heine wrote:
>> On 2018-05-29 10:20, [ext] Claudius Heine wrote:
>>>>>   # SPDX-License-Identifier: MIT
>>>>> +IMAGE_CFG_PACKAGE ??= ""
>>>>
>>>> Are you going to rename this variable to IMAGE_CFG_PACKAGES
>>>> and add a comment it in a separate patch?
>>>
>>> Oh, sorry. Forgot about that. I will prepare v5.
>>
>> I have thought on this and I think that 'IMAGE_CFG_PACKAGES' is a bad 
>> variable name, since it implies to be the variable to put all 
>> configuration packages into.
>> How about 'IMAGE_ONESHOT_PACKAGES', 'IMAGE_VOLATILE_PACKAGES' or 
>> 'IMAGE_TEMPORARY_PACKAGES'?
> 
> The purpose of these packages is to do image post configuration (or 
> tweaking)...
> May be IMAGE_POSTCONFIG_PACKAGES or IMAGE_TWEAK_PACKAGES?

Thanks for your suggestions!
I started with `IMAGE_TRANSIENT_PACKAGES` and quite like that.

My problem with names like `IMAGE_CFG_PACKAGES` or your suggestions is 
that it explains what kind of packages might make sense to be put there 
instead of how those packages are handled.

This variable is not the right one for all configuration packages, since 
a configuration package might deploy a configuration file not in a 
package script, but part of the package files. Then this file would be 
removed from the image it that package was added to this variable.

So instead I try to explain what will happen to those packages. "one 
shot" doesn't explain it good enough, while 'volatile' and 'temporary' 
might be misunderstood. 'Transient' however explains quite well IMO.

regards,
Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de