Re: [RFC PATCH 1/1] add minimization image extension

[Henning Schild <henning.schild@siemens.com>]

Hey,

looks good! I once had something similar but much less flexible in a
transient packages somehwere, uninstalling apt and removing a ton of
files.

I did use "ncdu" to identify files to look at. And there is
https://wiki.debian.org/ReduceDebian with some more ideas.

I guess one additional harsh but effective step could be deleting kernel
modules and rebuilding depmod. And if an initrd is used, rebuild that
as well.

Henning

Am Wed, 29 May 2019 12:45:06 +0200
schrieb "[ext] claudius.heine.ext@siemens.com"
<claudius.heine.ext@siemens.com>:

> From: Claudius Heine <ch@denx.de>
> 
> In case storage space on the device is limited, it is necessary to
> remove all packages and files that are not strictly needed for the
> device to perform its function.
> 
> This minimization feature allows to remove packages and files after
> the creation of the image and possible break the package management.
> 
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
>  .../recipes-core/images/isar-image-ubi.bb     |  2 +
>  .../image-minimizing-extension.bbclass        | 80
> +++++++++++++++++++ meta/classes/image.bbclass                    |
> 1 + meta/classes/rootfs.bbclass                   |  3 +-
>  4 files changed, 85 insertions(+), 1 deletion(-)
>  create mode 100644 meta/classes/image-minimizing-extension.bbclass
> 
> diff --git a/meta-isar/recipes-core/images/isar-image-ubi.bb
> b/meta-isar/recipes-core/images/isar-image-ubi.bb index
> 3b41f23..2971edf 100644 ---
> a/meta-isar/recipes-core/images/isar-image-ubi.bb +++
> b/meta-isar/recipes-core/images/isar-image-ubi.bb @@ -24,3 +24,5 @@
> DTB_IMG = "${PP_DEPLOY}/${DTB_FILE}" 
>  UBIFS_IMG = "${PP_DEPLOY}/${UBIFS_IMAGE_FILE}"
>  FIT_IMG = "${PP_DEPLOY}/${FIT_IMAGE_FILE}"
> +
> +IMAGE_MINIMIZATION_FEATURES += "remove-bug remove-bash-completion
> remove-manpages remove-info remove-lintian remove-locale remove-apt
> remove-dpkg remove-doc remove-licenses" diff --git
> a/meta/classes/image-minimizing-extension.bbclass
> b/meta/classes/image-minimizing-extension.bbclass new file mode
> 100644 index 0000000..f26f545 --- /dev/null +++
> b/meta/classes/image-minimizing-extension.bbclass @@ -0,0 +1,80 @@
> +# This software is a part of ISAR.
> +# Copyright (C) Siemens AG, 2019
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +# This class extends the image.bbclass for minimizing the root file
> system +
> +# Features:
> +# remove-bug
> +# remove-bash-completion
> +# remove-manpages
> +# remove-info
> +# remove-lintian
> +# remove-locale
> +# remove-apt
> +# remove-dpkg
> +# remove-doc
> +# remove-licenses
> +IMAGE_MINIMIZATION_FEATURES ?= ""
> +
> +IMAGE_MINIMIZATION_PACKAGES ?= ""
> +IMAGE_MINIMIZATION_FILES ?= ""
> +
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-bug',
> '/usr/share/bug', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-bash-completion', '/usr/share/bash-completion', '', d)}"
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-manpages', '/usr/share/man /usr/share/man-db', '', d)}"
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-info',
> '/usr/share/info', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-lintian', '/usr/share/lintian', '', d)}"
> +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-locale',
> '/usr/share/locale', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt',
> '/var/cache/apt /var/lib/apt /etc/apt', '', d)}"
> +IMAGE_MINIMIZATION_PACKAGES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-apt',
> d.getVar('IMAGE_MINIMIZATION_PACKAGES_APT'), '', d)}"
> +IMAGE_MINIMIZATION_PACKAGES_APT = 'apt libapt-pkg5.0
> debian-archive-keyring' +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg',
> '/var/lib/dpkg /etc/dpkg /etc/dpkg-cross', '', d)}"
> +IMAGE_MINIMIZATION_PACKAGES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-dpkg',
> 'dpkg', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES', 'remove-doc',
> '/usr/share/doc-base', '', d)}" +IMAGE_MINIMIZATION_FILES +=
> "${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-licenses', '/usr/share/common-licenses', '', d)}" +
> +ROOTFS_INSTALL_COMMAND += "image_install_minimization"
> +image_install_minimization[weight] = "5"
> +image_install_minimization() {
> +    sudo rm -rf '${WORKDIR}/dpkg-admin'
> +    sudo cp -a '${ROOTFSDIR}/var/lib/dpkg' '${WORKDIR}/dpkg-admin'
> +}
> +
> +ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_minimization"
> +
> +image_postprocess_minimization() {
> +    image_do_mounts
> +
> +    sudo mkdir -p '${BUILDROOT_ROOTFS}/dpkg-admin'
> +    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' || \
> +        sudo mount --bind '${WORKDIR}/dpkg-admin'
> '${BUILDROOT_ROOTFS}/dpkg-admin' +
> +    for package in ${IMAGE_MINIMIZATION_PACKAGES}; do
> +            sudo chroot ${BUILDCHROOT_DIR} /usr/bin/dpkg \
> +                    --root='${PP_ROOTFS}' \
> +                    --admindir='${PP_ROOTFS}/dpkg-admin' \
> +                    --force-remove-essential --force-depends -P
> $package
> +    done
> +
> +    mountpoint -q '${BUILDROOT_ROOTFS}/dpkg-admin' &&
> +        sudo umount -l '${BUILDROOT_ROOTFS}/dpkg-admin'
> +    sudo rmdir '${BUILDROOT_ROOTFS}/dpkg-admin'
> +
> +    for file in ${IMAGE_MINIMIZATION_FILES}; do
> +        sudo rm -rf '${IMAGE_ROOTFS}/'"$file"
> +    done
> +
> +    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-doc', '1', '', d)}' ]; then
> +        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
> +        \! -regex
> '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*'
> \
> +        -delete
> +    fi
> +
> +    if [ -n '${@bb.utils.contains('IMAGE_MINIMIZATION_FEATURES',
> 'remove-licenses', '1', '', d)}' ]; then
> +        sudo find '${IMAGE_ROOTFS}/usr/share/doc' -type f \
> +        -regex
> '${IMAGE_ROOTFS}/usr/share/doc/[^/]+/\(copyright\|CREDITS\|AUTHORS\).*'
> \
> +        -delete
> +    fi
> +}
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 5682134..681cdc4 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -67,6 +67,7 @@ inherit image-tools-extension
>  inherit image-postproc-extension
>  inherit image-locales-extension
>  inherit image-account-extension
> +inherit image-minimizing-extension
>  
>  # Extra space for rootfs in MB
>  ROOTFS_EXTRA ?= "64"
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index c7e0435..59a431e 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -205,7 +205,8 @@ rootfs_postprocess_finalize() {
>          rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
>          rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
>  
> -        mv "${ROOTFSDIR}/etc/apt/sources-list" \
> +        test -e "${ROOTFSDIR}/etc/apt/sources-list" && \
> +            mv "${ROOTFSDIR}/etc/apt/sources-list" \
>              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
>  
>          rm -f "${ROOTFSDIR}/etc/apt/sources-list"