[RFC PATCH 0/1] Image minimization

[RFC PATCH 0/1] Image minimization

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Hi,

here is a RFC patch that adds a minimization extensions to images.

The basic idea behind this is that it allows to remove specific packages and
files from the root file system in a post-processing step. This is useful in
case there are tight storage space restrictions. It breaks the package
management in multiple different ways and degrees, but the main idea is to
reduce the root file system to a minimal set of files that are necessary for the
device to still perform its function.

I envision the `IMAGE_MINIMIZATION_FEATURES`, `IMAGE_MINIMIZATION_PACKAGES` and
`IMAGE_MINIMIZATION_FILES` to be the user facing interface for this.

While removing packages, I use the dpkg from the buildchroot in order to be
independent from the tools on the image. This also allows to remove dpkg while
still being able to use it.

I would like to hear some input about the general design, before I start
cleaning that up and testing it more intensively.

thanks,
Claudius

Claudius Heine (1):
  add minimization image extension

 .../recipes-core/images/isar-image-ubi.bb     |  2 +
 .../image-minimizing-extension.bbclass        | 80 +++++++++++++++++++
 meta/classes/image.bbclass                    |  1 +
 meta/classes/rootfs.bbclass                   |  3 +-
 4 files changed, 85 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes/image-minimizing-extension.bbclass

-- 
2.20.1

[RFC PATCH 1/1] add minimization image extension

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

In case storage space on the device is limited, it is necessary to
remove all packages and files that are not strictly needed for the
device to perform its function.

This minimization feature allows to remove packages and files after the
creation of the image and possible break the package management.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 .../recipes-core/images/isar-image-ubi.bb     |  2 +
 .../image-minimizing-extension.bbclass        | 80 +++++++++++++++++++
 meta/classes/image.bbclass                    |  1 +
 meta/classes/rootfs.bbclass                   |  3 +-
 4 files changed, 85 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes/image-minimizing-extension.bbclass

diff --git a/meta-isar/recipes-core/images/isar-image-ubi.bb b/meta-isar/recipes-core/images/isar-image-ubi.bb
index 3b41f23..2971edf 100644
--- a/meta-isar/recipes-core/images/isar-image-ubi.bb
+++ b/meta-isar/recipes-core/images/isar-image-ubi.bb
@@ -24,3 +24,5 @@ DTB_IMG = "${PP_DEPLOY}/${DTB_FILE}"
 
 UBIFS_IMG = "${PP_DEPLOY}/${UBIFS_IMAGE_FILE}"
 FIT_IMG = "${PP_DEPLOY}/${FIT_IMAGE_FILE}"
+
+IMAGE_MINIMIZATION_FEATURES += "remove-bug remove-bash-completion remove-manpages remove-info remove-lintian remove-locale remove-apt remove-dpkg remove-doc remove-licenses"
diff --git a/meta/classes/image-minimizing-extension.bbclass b/meta/classes/image-minimizing-extension.bbclass
new file mode 100644
index 0000000..f26f545
--- /dev/null
+++ b/meta/classes/image-minimizing-extension.bbclass
@@ -0,0 +1,80 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass for minimizing the root file system
+
+# Features:
+# remove-bug
+# remove-bash-completion
+# remove-manpages
+# remove-info
+# remove-lintian
+# remove-locale
+# remove-apt
+# remove-dpkg
+# remove-doc
+# remove-licenses
+IMAGE_MINIMIZATION_FEATURES ?= ""
+
+IMAGE_MINIMIZATION_PACKAGES ?= ""
+IMAGE_MINIMIZATION_FILES ?= ""
+
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-bug', '/usr/share/bug', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-bash-completion', '/usr/share/bash-completion', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-manpages', '/usr/share/man /usr/share/man-db', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-info', '/usr/share/info', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-lintian', '/usr/share/lintian', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-locale', '/usr/share/locale', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt', '/var/cache/apt /var/lib/apt /etc/apt', '', d)}"
+IMAGE_MINIMIZATION_PACKAGES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt', d.getVar('IMAGE_MINIMIZATION_PACKAGES_APT'), '', d)}"
+IMAGE_MINIMIZATION_PACKAGES_APT = 'apt libapt-pkg5.0 debian-archive-keyring'
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg', '/var/lib/dpkg /etc/dpkg /etc/dpkg-cross', '', d)}"
+IMAGE_MINIMIZATION_PACKAGES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg', 'dpkg', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-doc', '/usr/share/doc-base', '', d)}"
+IMAGE_MINIMIZATION_FILES += "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-licenses', '/usr/share/common-licenses', '', d)}"
+
+ROOTFS_INSTALL_COMMAND += "image_install_minimization"
+image_install_minimization[weight] = "5"
+image_install_minimization() {
+    sudo rm -rf '${WORKDIR}/dpkg-admin'
+    sudo cp -a '${ROOTFSDIR}/var/lib/dpkg' '${WORKDIR}/dpkg-admin'
+}
+
+ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_minimization"
+
+image_postprocess_minimization() {
+    image_do_mounts
+
+    sudo mkdir -p '${BUILDROOT_ROOTFS}/dpkg-admin'
+    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' || \
+        sudo mount --bind '${WORKDIR}/dpkg-admin' '${BUILDROOT_ROOTFS}/dpkg-admin'
+
+    for package in ${IMAGE_MINIMIZATION_PACKAGES}; do
+            sudo chroot ${BUILDCHROOT_DIR} /usr/bin/dpkg \
+                    --root='${PP_ROOTFS}' \
+                    --admindir='${PP_ROOTFS}/dpkg-admin' \
+                    --force-remove-essential --force-depends -P $package
+    done
+
+    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' &&
+        sudo umount -l '${BUILDROOT_ROOTFS}/dpkg-admin'
+    sudo rmdir '${BUILDROOT_ROOTFS}/dpkg-admin'
+
+    for file in ${IMAGE_MINIMIZATION_FILES}; do
+        sudo rm -rf '${IMAGE_ROOTFS}/'"$file"
+    done
+
+    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-doc', '1', '', d)}' ]; then
+        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
+        \! -regex '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*' \
+        -delete
+    fi
+
+    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-licenses', '1', '', d)}' ]; then
+        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
+        -regex '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*' \
+        -delete
+    fi
+}
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 5682134..681cdc4 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -67,6 +67,7 @@ inherit image-tools-extension
 inherit image-postproc-extension
 inherit image-locales-extension
 inherit image-account-extension
+inherit image-minimizing-extension
 
 # Extra space for rootfs in MB
 ROOTFS_EXTRA ?= "64"
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index c7e0435..59a431e 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -205,7 +205,8 @@ rootfs_postprocess_finalize() {
         rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
         rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
 
-        mv "${ROOTFSDIR}/etc/apt/sources-list" \
+        test -e "${ROOTFSDIR}/etc/apt/sources-list" && \
+            mv "${ROOTFSDIR}/etc/apt/sources-list" \
             "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
 
         rm -f "${ROOTFSDIR}/etc/apt/sources-list"
-- 
2.20.1

Re: [RFC PATCH 0/1] Image minimization

[Jan Kiszka]

On 29.05.19 12:45, [ext] claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
> 
> Hi,
> 
> here is a RFC patch that adds a minimization extensions to images.
> 
> The basic idea behind this is that it allows to remove specific packages and
> files from the root file system in a post-processing step. This is useful in
> case there are tight storage space restrictions. It breaks the package
> management in multiple different ways and degrees, but the main idea is to
> reduce the root file system to a minimal set of files that are necessary for the
> device to still perform its function.
> 
> I envision the `IMAGE_MINIMIZATION_FEATURES`, `IMAGE_MINIMIZATION_PACKAGES` and
> `IMAGE_MINIMIZATION_FILES` to be the user facing interface for this.
> 
> While removing packages, I use the dpkg from the buildchroot in order to be
> independent from the tools on the image. This also allows to remove dpkg while
> still being able to use it.
> 
> I would like to hear some input about the general design, before I start
> cleaning that up and testing it more intensively.

Looks reasonable from the interface POV. So, users would normally only juggle
with our predefined minimizations features and, thus,
IMAGE_MINIMIZATION_FEATURES. Only very few cases would then require stepping one
level down and fiddle directly with package lists or even individual files.
Would likely make this reasonably convenient.

I would suggest adding a tiny demo image as well, something that only gives you
a shell prompt or so.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [RFC PATCH 1/1] add minimization image extension

[Henning Schild]

Hey,

looks good! I once had something similar but much less flexible in a
transient packages somehwere, uninstalling apt and removing a ton of
files.

I did use "ncdu" to identify files to look at. And there is
https://wiki.debian.org/ReduceDebian with some more ideas.

I guess one additional harsh but effective step could be deleting kernel
modules and rebuilding depmod. And if an initrd is used, rebuild that
as well.

Henning

Am Wed, 29 May 2019 12:45:06 +0200
schrieb "[ext] claudius.heine.ext@siemens.com"
<claudius.heine.ext@siemens.com>:

> From: Claudius Heine <ch@denx.de>
> 
> In case storage space on the device is limited, it is necessary to
> remove all packages and files that are not strictly needed for the
> device to perform its function.
> 
> This minimization feature allows to remove packages and files after
> the creation of the image and possible break the package management.
> 
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
>  .../recipes-core/images/isar-image-ubi.bb     |  2 +
>  .../image-minimizing-extension.bbclass        | 80
> +++++++++++++++++++ meta/classes/image.bbclass                    |
> 1 + meta/classes/rootfs.bbclass                   |  3 +-
>  4 files changed, 85 insertions(+), 1 deletion(-)
>  create mode 100644 meta/classes/image-minimizing-extension.bbclass
> 
> diff --git a/meta-isar/recipes-core/images/isar-image-ubi.bb
> b/meta-isar/recipes-core/images/isar-image-ubi.bb index
> 3b41f23..2971edf 100644 ---
> a/meta-isar/recipes-core/images/isar-image-ubi.bb +++
> b/meta-isar/recipes-core/images/isar-image-ubi.bb @@ -24,3 +24,5 @@
> DTB_IMG = "${PP_DEPLOY}/${DTB_FILE}" 
>  UBIFS_IMG = "${PP_DEPLOY}/${UBIFS_IMAGE_FILE}"
>  FIT_IMG = "${PP_DEPLOY}/${FIT_IMAGE_FILE}"
> +
> +IMAGE_MINIMIZATION_FEATURES += "remove-bug remove-bash-completion
> remove-manpages remove-info remove-lintian remove-locale remove-apt
> remove-dpkg remove-doc remove-licenses" diff --git
> a/meta/classes/image-minimizing-extension.bbclass
> b/meta/classes/image-minimizing-extension.bbclass new file mode
> 100644 index 0000000..f26f545 --- /dev/null +++
> b/meta/classes/image-minimizing-extension.bbclass @@ -0,0 +1,80 @@
> +# This software is a part of ISAR.
> +# Copyright (C) Siemens AG, 2019
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +# This class extends the image.bbclass for minimizing the root file
> system +
> +# Features:
> +# remove-bug
> +# remove-bash-completion
> +# remove-manpages
> +# remove-info
> +# remove-lintian
> +# remove-locale
> +# remove-apt
> +# remove-dpkg
> +# remove-doc
> +# remove-licenses
> +IMAGE_MINIMIZATION_FEATURES ?= ""
> +
> +IMAGE_MINIMIZATION_PACKAGES ?= ""
> +IMAGE_MINIMIZATION_FILES ?= ""
> +
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-bug',
> '/usr/share/bug', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-bash-completion', '/usr/share/bash-completion', '', d)}"
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-manpages', '/usr/share/man /usr/share/man-db', '', d)}"
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-info',
> '/usr/share/info', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-lintian', '/usr/share/lintian', '', d)}"
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-locale',
> '/usr/share/locale', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt',
> '/var/cache/apt /var/lib/apt /etc/apt', '', d)}"
> +IMAGE_MINIMIZATION_PACKAGES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt',
> d.getVar('IMAGE_MINIMIZATION_PACKAGES_APT'), '', d)}"
> +IMAGE_MINIMIZATION_PACKAGES_APT = 'apt libapt-pkg5.0
> debian-archive-keyring' +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg',
> '/var/lib/dpkg /etc/dpkg /etc/dpkg-cross', '', d)}"
> +IMAGE_MINIMIZATION_PACKAGES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg',
> 'dpkg', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-doc',
> '/usr/share/doc-base', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-licenses', '/usr/share/common-licenses', '', d)}" +
> +ROOTFS_INSTALL_COMMAND += "image_install_minimization"
> +image_install_minimization[weight] = "5"
> +image_install_minimization() {
> +    sudo rm -rf '${WORKDIR}/dpkg-admin'
> +    sudo cp -a '${ROOTFSDIR}/var/lib/dpkg' '${WORKDIR}/dpkg-admin'
> +}
> +
> +ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_minimization"
> +
> +image_postprocess_minimization() {
> +    image_do_mounts
> +
> +    sudo mkdir -p '${BUILDROOT_ROOTFS}/dpkg-admin'
> +    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' || \
> +        sudo mount --bind '${WORKDIR}/dpkg-admin'
> '${BUILDROOT_ROOTFS}/dpkg-admin' +
> +    for package in ${IMAGE_MINIMIZATION_PACKAGES}; do
> +            sudo chroot ${BUILDCHROOT_DIR} /usr/bin/dpkg \
> +                    --root='${PP_ROOTFS}' \
> +                    --admindir='${PP_ROOTFS}/dpkg-admin' \
> +                    --force-remove-essential --force-depends -P
> $package
> +    done
> +
> +    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' &&
> +        sudo umount -l '${BUILDROOT_ROOTFS}/dpkg-admin'
> +    sudo rmdir '${BUILDROOT_ROOTFS}/dpkg-admin'
> +
> +    for file in ${IMAGE_MINIMIZATION_FILES}; do
> +        sudo rm -rf '${IMAGE_ROOTFS}/'"$file"
> +    done
> +
> +    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-doc', '1', '', d)}' ]; then
> +        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
> +        \! -regex
> '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*'
> \
> +        -delete
> +    fi
> +
> +    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-licenses', '1', '', d)}' ]; then
> +        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
> +        -regex
> '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*'
> \
> +        -delete
> +    fi
> +}
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 5682134..681cdc4 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -67,6 +67,7 @@ inherit image-tools-extension
>  inherit image-postproc-extension
>  inherit image-locales-extension
>  inherit image-account-extension
> +inherit image-minimizing-extension
>  
>  # Extra space for rootfs in MB
>  ROOTFS_EXTRA ?= "64"
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index c7e0435..59a431e 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -205,7 +205,8 @@ rootfs_postprocess_finalize() {
>          rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
>          rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
>  
> -        mv "${ROOTFSDIR}/etc/apt/sources-list" \
> +        test -e "${ROOTFSDIR}/etc/apt/sources-list" && \
> +            mv "${ROOTFSDIR}/etc/apt/sources-list" \
>              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
>  
>          rm -f "${ROOTFSDIR}/etc/apt/sources-list"

Re: [RFC PATCH 1/1] add minimization image extension

[Henning Schild]

Am Wed, 29 May 2019 13:22:15 +0200
schrieb "[ext] Henning Schild" <henning.schild@siemens.com>:

> Hey,
> 
> looks good! I once had something similar but much less flexible in a
> transient packages somehwere, uninstalling apt and removing a ton of
> files.
> 
> I did use "ncdu" to identify files to look at. And there is
> https://wiki.debian.org/ReduceDebian with some more ideas.
> 
> I guess one additional harsh but effective step could be deleting
> kernel modules and rebuilding depmod. And if an initrd is used,
> rebuild that as well.

This one probably is not even worth the effort, because such restricted
devices are rare and will die out soon (fingers crossed). And they will
likely get a custom kernel anyways.

But i would suggest the demo image to use a custom minimal kernel. So
we can just "du -sh" that rootfs when someone asks how small you can
get.

Henning

> Henning
> 
> Am Wed, 29 May 2019 12:45:06 +0200
> schrieb "[ext] claudius.heine.ext@siemens.com"
> <claudius.heine.ext@siemens.com>:
> 
> > From: Claudius Heine <ch@denx.de>
> > 
> > In case storage space on the device is limited, it is necessary to
> > remove all packages and files that are not strictly needed for the
> > device to perform its function.
> > 
> > This minimization feature allows to remove packages and files after
> > the creation of the image and possible break the package management.
> > 
> > Signed-off-by: Claudius Heine <ch@denx.de>
> > ---
> >  .../recipes-core/images/isar-image-ubi.bb     |  2 +
> >  .../image-minimizing-extension.bbclass        | 80
> > +++++++++++++++++++ meta/classes/image.bbclass                    |
> > 1 + meta/classes/rootfs.bbclass                   |  3 +-
> >  4 files changed, 85 insertions(+), 1 deletion(-)
> >  create mode 100644 meta/classes/image-minimizing-extension.bbclass
> > 
> > diff --git a/meta-isar/recipes-core/images/isar-image-ubi.bb
> > b/meta-isar/recipes-core/images/isar-image-ubi.bb index
> > 3b41f23..2971edf 100644 ---
> > a/meta-isar/recipes-core/images/isar-image-ubi.bb +++
> > b/meta-isar/recipes-core/images/isar-image-ubi.bb @@ -24,3 +24,5 @@
> > DTB_IMG = "${PP_DEPLOY}/${DTB_FILE}" 
> >  UBIFS_IMG = "${PP_DEPLOY}/${UBIFS_IMAGE_FILE}"
> >  FIT_IMG = "${PP_DEPLOY}/${FIT_IMAGE_FILE}"
> > +
> > +IMAGE_MINIMIZATION_FEATURES += "remove-bug remove-bash-completion
> > remove-manpages remove-info remove-lintian remove-locale remove-apt
> > remove-dpkg remove-doc remove-licenses" diff --git
> > a/meta/classes/image-minimizing-extension.bbclass
> > b/meta/classes/image-minimizing-extension.bbclass new file mode
> > 100644 index 0000000..f26f545 --- /dev/null +++
> > b/meta/classes/image-minimizing-extension.bbclass @@ -0,0 +1,80 @@
> > +# This software is a part of ISAR.
> > +# Copyright (C) Siemens AG, 2019
> > +#
> > +# SPDX-License-Identifier: MIT
> > +#
> > +# This class extends the image.bbclass for minimizing the root file
> > system +
> > +# Features:
> > +# remove-bug
> > +# remove-bash-completion
> > +# remove-manpages
> > +# remove-info
> > +# remove-lintian
> > +# remove-locale
> > +# remove-apt
> > +# remove-dpkg
> > +# remove-doc
> > +# remove-licenses
> > +IMAGE_MINIMIZATION_FEATURES ?= ""
> > +
> > +IMAGE_MINIMIZATION_PACKAGES ?= ""
> > +IMAGE_MINIMIZATION_FILES ?= ""
> > +
> > +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-bug',
> > '/usr/share/bug', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-bash-completion', '/usr/share/bash-completion', '', d)}"
> > +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-manpages', '/usr/share/man /usr/share/man-db', '', d)}"
> > +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-info',
> > '/usr/share/info', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-lintian', '/usr/share/lintian', '', d)}"
> > +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-locale', '/usr/share/locale', '', d)}"
> > +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt',
> > '/var/cache/apt /var/lib/apt /etc/apt', '', d)}"
> > +IMAGE_MINIMIZATION_PACKAGES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt',
> > d.getVar('IMAGE_MINIMIZATION_PACKAGES_APT'), '', d)}"
> > +IMAGE_MINIMIZATION_PACKAGES_APT = 'apt libapt-pkg5.0
> > debian-archive-keyring' +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg',
> > '/var/lib/dpkg /etc/dpkg /etc/dpkg-cross', '', d)}"
> > +IMAGE_MINIMIZATION_PACKAGES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg',
> > 'dpkg', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-doc',
> > '/usr/share/doc-base', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> > "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-licenses', '/usr/share/common-licenses', '', d)}" +
> > +ROOTFS_INSTALL_COMMAND += "image_install_minimization"
> > +image_install_minimization[weight] = "5"
> > +image_install_minimization() {
> > +    sudo rm -rf '${WORKDIR}/dpkg-admin'
> > +    sudo cp -a '${ROOTFSDIR}/var/lib/dpkg' '${WORKDIR}/dpkg-admin'
> > +}
> > +
> > +ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_minimization"
> > +
> > +image_postprocess_minimization() {
> > +    image_do_mounts
> > +
> > +    sudo mkdir -p '${BUILDROOT_ROOTFS}/dpkg-admin'
> > +    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' || \
> > +        sudo mount --bind '${WORKDIR}/dpkg-admin'
> > '${BUILDROOT_ROOTFS}/dpkg-admin' +
> > +    for package in ${IMAGE_MINIMIZATION_PACKAGES}; do
> > +            sudo chroot ${BUILDCHROOT_DIR} /usr/bin/dpkg \
> > +                    --root='${PP_ROOTFS}' \
> > +                    --admindir='${PP_ROOTFS}/dpkg-admin' \
> > +                    --force-remove-essential --force-depends -P
> > $package
> > +    done
> > +
> > +    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' &&
> > +        sudo umount -l '${BUILDROOT_ROOTFS}/dpkg-admin'
> > +    sudo rmdir '${BUILDROOT_ROOTFS}/dpkg-admin'
> > +
> > +    for file in ${IMAGE_MINIMIZATION_FILES}; do
> > +        sudo rm -rf '${IMAGE_ROOTFS}/'"$file"
> > +    done
> > +
> > +    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-doc', '1', '', d)}' ]; then
> > +        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
> > +        \! -regex
> > '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*'
> > \
> > +        -delete
> > +    fi
> > +
> > +    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> > 'remove-licenses', '1', '', d)}' ]; then
> > +        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
> > +        -regex
> > '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*'
> > \
> > +        -delete
> > +    fi
> > +}
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > index 5682134..681cdc4 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -67,6 +67,7 @@ inherit image-tools-extension
> >  inherit image-postproc-extension
> >  inherit image-locales-extension
> >  inherit image-account-extension
> > +inherit image-minimizing-extension
> >  
> >  # Extra space for rootfs in MB
> >  ROOTFS_EXTRA ?= "64"
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass index c7e0435..59a431e 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -205,7 +205,8 @@ rootfs_postprocess_finalize() {
> >          rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
> >          rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
> >  
> > -        mv "${ROOTFSDIR}/etc/apt/sources-list" \
> > +        test -e "${ROOTFSDIR}/etc/apt/sources-list" && \
> > +            mv "${ROOTFSDIR}/etc/apt/sources-list" \
> >              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
> >  
> >          rm -f "${ROOTFSDIR}/etc/apt/sources-list"  
> 

Re: [RFC PATCH 1/1] add minimization image extension

[Jan Kiszka]

On 29.05.19 13:22, [ext] Henning Schild wrote:
> Hey,
> 
> looks good! I once had something similar but much less flexible in a
> transient packages somehwere, uninstalling apt and removing a ton of
> files.
> 
> I did use "ncdu" to identify files to look at. And there is
> https://wiki.debian.org/ReduceDebian with some more ideas.
> 
> I guess one additional harsh but effective step could be deleting kernel
> modules and rebuilding depmod. And if an initrd is used, rebuild that
> as well.

Well, kernel modules is a topic for custom kernels - if you customize, you don't
need to delete. But deleting initrds is indeed a nice reusable pattern. Most
systems with custom kernels have no need for them anyway.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux