[RFC 0/2] Remove Packages during Postprocessing

[RFC 0/2] Remove Packages during Postprocessing

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

Some packages even if the are part of minbase are not necessary
to run a debian system. Debian has some issues and experiments
to remove packages from minbase [1]. This feature allows a
expert user to remove packages from the final image during post
processing.

The reason for this are e.g. disk usage reduction and reduction
of the clearing effort.

The method to remove packages in postprocessing is a best-effort
action.

Another way would be to identify like [1] packages in
minbase which can be removed without compremising the isar rootfs
creation and remove them directly after or during bootstrapping.
If a package is used for a production related feature it should be
reinstalled.


[1]: https://wiki.debian.org/BusterPriorityRequalification

Quirin Gylstorff (2):
  meta/classes: Add remove packages to rootfs postprocessing
  meta-isar/images: Remove gcc-8-base from rootfs

 meta-isar/recipes-core/images/isar-image-base.bb |  4 ++++
 meta/classes/image.bbclass                       |  2 +-
 meta/classes/rootfs.bbclass                      | 13 +++++++++++++
 3 files changed, 18 insertions(+), 1 deletion(-)

-- 
2.20.1

[RFC 1/2] meta/classes: Add remove packages to rootfs postprocessing

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

Remove non-essential packages to reduce the amount of packages
to be cleared. It can also to used to reduce the foot print
of a image.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 meta/classes/image.bbclass  |  2 +-
 meta/classes/rootfs.bbclass | 13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 6bcc6cd..62d6f46 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -60,7 +60,7 @@ image_do_mounts() {
 }
 
 ROOTFSDIR = "${IMAGE_ROOTFS}"
-ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs generate-manifest"
+ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs generate-manifest remove_packages"
 ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
 ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 64eaef7..0329f55 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -187,6 +187,19 @@ rootfs_postprocess_clean_package_cache() {
         /usr/bin/apt-get clean
     sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
 }
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'remove_packages', 'rootfs_rm_packages', '', d)}"
+rootfs_rm_packages () {
+    if [ -n "${ROOTFS_PACKAGES_FORCE_REMOVE}" ]; then
+        RMPKGS=$(sudo -E chroot "${ROOTFSDIR}" \
+              dpkg-query \
+              -f '${Package}\n' \
+              -W ${ROOTFS_PACKAGES_FORCE_REMOVE} 2>/dev/null || true)
+    fi
+    if [ -n "${RMPKGS}" ]; then
+        sudo -E chroot "${ROOTFSDIR}" \
+            dpkg --purge --force-depends ${RMPKGS}
+    fi
+}
 
 ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'rootfs_generate_manifest', '', d)}"
 rootfs_generate_manifest () {
-- 
2.20.1

Re: [RFC 1/2] meta/classes: Add remove packages to rootfs postprocessing

[Jan Kiszka]

On 21.02.20 15:53, Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> Remove non-essential packages to reduce the amount of packages
> to be cleared. It can also to used to reduce the foot print
> of a image.

Lacks documentation: This does not remove non-essential packages by 
itself. It provides a configuration channel 
(ROOTFS_PACKAGES_FORCE_REMOVE) to request arbitrary package removals. 
That should be clearly described, along with the warning about this 
weapon (expert tool).

Do you also have a set of removal candidates to share with users?

> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> ---
>   meta/classes/image.bbclass  |  2 +-
>   meta/classes/rootfs.bbclass | 13 +++++++++++++
>   2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 6bcc6cd..62d6f46 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -60,7 +60,7 @@ image_do_mounts() {
>   }
>   
>   ROOTFSDIR = "${IMAGE_ROOTFS}"
> -ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs generate-manifest"
> +ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs generate-manifest remove_packages"
>   ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
>   ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
>   
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index 64eaef7..0329f55 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -187,6 +187,19 @@ rootfs_postprocess_clean_package_cache() {
>           /usr/bin/apt-get clean
>       sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
>   }
> +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'remove_packages', 'rootfs_rm_packages', '', d)}"
> +rootfs_rm_packages () {
> +    if [ -n "${ROOTFS_PACKAGES_FORCE_REMOVE}" ]; then
> +        RMPKGS=$(sudo -E chroot "${ROOTFSDIR}" \
> +              dpkg-query \
> +              -f '${Package}\n' \
> +              -W ${ROOTFS_PACKAGES_FORCE_REMOVE} 2>/dev/null || true)
> +    fi
> +    if [ -n "${RMPKGS}" ]; then
> +        sudo -E chroot "${ROOTFSDIR}" \
> +            dpkg --purge --force-depends ${RMPKGS}
> +    fi
> +}
>   
>   ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'rootfs_generate_manifest', '', d)}"
>   rootfs_generate_manifest () {
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [RFC 1/2] meta/classes: Add remove packages to rootfs postprocessing

[Gylstorff Quirin]

On 2/21/20 7:28 PM, Jan Kiszka wrote:
> On 21.02.20 15:53, Q. Gylstorff wrote:
>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>>
>> Remove non-essential packages to reduce the amount of packages
>> to be cleared. It can also to used to reduce the foot print
>> of a image.
> 
> Lacks documentation: This does not remove non-essential packages by 
> itself. It provides a configuration channel 
> (ROOTFS_PACKAGES_FORCE_REMOVE) to request arbitrary package removals. 
> That should be clearly described, along with the warning about this 
> weapon (expert tool).
> 
> Do you also have a set of removal candidates to share with users?
>

You mean something like this:

This function allows to remove non essential debian packages
even if there are dependencies to them.

The feature can be activate in a image or sdk rootfs by adding
`remove_packages` to the variable ROOTFS_FEATURES.

The packages you want to remove are added to
`ROOTFS_PACKAGES_FORCE_REMOVE` separated by a space. e.g.
ROOTFS_PACKAGES_FORCE_REMOVE = "example_package".

The following package are installed and are non-essential [1]:
adduser, apt, debian-archive-keyring, gcc-8-base, gpgv,
libapt-pkg5.0, libffi6, libgmp10, libgnutls30, libhogweed4,
libidn2-0, libnettle6, libp11-kit0, libsemanage-common,
libsemanage1, libsepol1, libstdc++6, libtasn1-6, libunistring2,
libustr-1.0-1, lsb-base, passwd, tzdata

Also removable are:
e2fsprogs[2]

The removal of a package depends on the given use-case and
should be executed by a expert user as it can leave the
system non-functional.

[1]: https://wiki.debian.org/BusterPriorityRequalification
[2]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=474540




>>
>> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>> ---
>>   meta/classes/image.bbclass  |  2 +-
>>   meta/classes/rootfs.bbclass | 13 +++++++++++++
>>   2 files changed, 14 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 6bcc6cd..62d6f46 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -60,7 +60,7 @@ image_do_mounts() {
>>   }
>>   ROOTFSDIR = "${IMAGE_ROOTFS}"
>> -ROOTFS_FEATURES += "copy-package-cache clean-package-cache 
>> finalize-rootfs generate-manifest"
>> +ROOTFS_FEATURES += "copy-package-cache clean-package-cache 
>> finalize-rootfs generate-manifest remove_packages"
>>   ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
>>   ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
>> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
>> index 64eaef7..0329f55 100644
>> --- a/meta/classes/rootfs.bbclass
>> +++ b/meta/classes/rootfs.bbclass
>> @@ -187,6 +187,19 @@ rootfs_postprocess_clean_package_cache() {
>>           /usr/bin/apt-get clean
>>       sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
>>   }
>> +ROOTFS_POSTPROCESS_COMMAND += 
>> "${@bb.utils.contains('ROOTFS_FEATURES', 'remove_packages', 
>> 'rootfs_rm_packages', '', d)}"
>> +rootfs_rm_packages () {
>> +    if [ -n "${ROOTFS_PACKAGES_FORCE_REMOVE}" ]; then
>> +        RMPKGS=$(sudo -E chroot "${ROOTFSDIR}" \
>> +              dpkg-query \
>> +              -f '${Package}\n' \
>> +              -W ${ROOTFS_PACKAGES_FORCE_REMOVE} 2>/dev/null || true)
>> +    fi
>> +    if [ -n "${RMPKGS}" ]; then
>> +        sudo -E chroot "${ROOTFSDIR}" \
>> +            dpkg --purge --force-depends ${RMPKGS}
>> +    fi
>> +}
>>   ROOTFS_POSTPROCESS_COMMAND += 
>> "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 
>> 'rootfs_generate_manifest', '', d)}"
>>   rootfs_generate_manifest () {
>>
> 
> Jan
> 

-- 
Quirin Gylstorff

Siemens AG
Corporate Technology
Research in Digitalization and Automation
Smart Embedded Systems
CT RDA IOT SES-DE
Otto-Hahn-Ring 6
81739 Muenchen, Germany
Mobile: +49 173 3746683
mailto:quirin.gylstorff@siemens.com
www.siemens.com/ingenuityforlife

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim 
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and 
Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, 
Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered offices: Berlin 
and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 
12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322

Important notice: This e-mail and any attachment thereof contain 
corporate proprietary information. If you have received it by mistake, 
please notify us immediately by reply e-mail and delete this e-mail and 
its attachments from your system. Thank you.

[RFC 2/2] meta-isar/images: Remove gcc-8-base from rootfs

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This is a testcase for the new feature ROOTFS_PACKAGE_FORCE_REMOVE.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 meta-isar/recipes-core/images/isar-image-base.bb | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index b381d85..18f366e 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -11,3 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
 PV = "1.0"
 
 inherit image
+# remove some packages
+ROOTFS_PACKAGES_FORCE_REMOVE ?= " \
+gcc-8-base \
+"
-- 
2.20.1

Re: [RFC 2/2] meta-isar/images: Remove gcc-8-base from rootfs

[Jan Kiszka]

On 21.02.20 15:53, Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> This is a testcase for the new feature ROOTFS_PACKAGE_FORCE_REMOVE.
> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> ---
>   meta-isar/recipes-core/images/isar-image-base.bb | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
> index b381d85..18f366e 100644
> --- a/meta-isar/recipes-core/images/isar-image-base.bb
> +++ b/meta-isar/recipes-core/images/isar-image-base.bb
> @@ -11,3 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
>   PV = "1.0"
>   
>   inherit image
> +# remove some packages
> +ROOTFS_PACKAGES_FORCE_REMOVE ?= " \
> +gcc-8-base \
> +"
> 

Please indent properly.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [RFC 0/2] Remove Packages during Postprocessing

[Henning Schild]

Hi,

my opinion on that is clear. Fix it upstream or live with those
packages. You are either on a distro or fiddle around and tune
everything until you are the only one on the planet testing your setup.
That is Isar vs. yocto ... whoever thinks they _need_ that should maybe
think again. If they need it they can put it into their own layer or
use yocto ;).
I do not think upstream should carry such hacky features unless we get
better reasoning ... Removing "required" packages has the potential to
break your image in funny ways ... that is much more expensive than a
few MB disc space. All affected packages are likely already cleared and
vulnerabilty monitored by someone else, find that someone and share the
cost!

Henning

On Fri, 21 Feb 2020 15:53:46 +0100
"Q. Gylstorff" <Quirin.Gylstorff@siemens.com> wrote:

> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> Some packages even if the are part of minbase are not necessary
> to run a debian system. Debian has some issues and experiments
> to remove packages from minbase [1]. This feature allows a
> expert user to remove packages from the final image during post
> processing.
> 
> The reason for this are e.g. disk usage reduction and reduction
> of the clearing effort.
> 
> The method to remove packages in postprocessing is a best-effort
> action.
> 
> Another way would be to identify like [1] packages in
> minbase which can be removed without compremising the isar rootfs
> creation and remove them directly after or during bootstrapping.
> If a package is used for a production related feature it should be
> reinstalled.
> 
> 
> [1]: https://wiki.debian.org/BusterPriorityRequalification
> 
> Quirin Gylstorff (2):
>   meta/classes: Add remove packages to rootfs postprocessing
>   meta-isar/images: Remove gcc-8-base from rootfs
> 
>  meta-isar/recipes-core/images/isar-image-base.bb |  4 ++++
>  meta/classes/image.bbclass                       |  2 +-
>  meta/classes/rootfs.bbclass                      | 13 +++++++++++++
>  3 files changed, 18 insertions(+), 1 deletion(-)
> 

Re: [RFC 0/2] Remove Packages during Postprocessing

[Gylstorff Quirin]

On 2/24/20 3:24 PM, Henning Schild wrote:
> Hi,
> 
> my opinion on that is clear. Fix it upstream or live with those
> packages. You are either on a distro or fiddle around and tune
> everything until you are the only one on the planet testing your setup.
> That is Isar vs. yocto ... whoever thinks they _need_ that should maybe
> think again. If they need it they can put it into their own layer or
> use yocto ;).
> I do not think upstream should carry such hacky features unless we get
> better reasoning ... Removing "required" packages has the potential to
> break your image in funny ways ... that is much more expensive than a
> few MB disc space. All affected packages are likely already cleared and
> vulnerabilty monitored by someone else, find that someone and share the
> cost!
> 
> Henning

Hi Henning,

I understand your concern and I think you are right. But some people
already hack the build process in similar ways and this is a way to give
them some support.

I want to collect the option of community regarding this patch.

Quirin

> 
> On Fri, 21 Feb 2020 15:53:46 +0100
> "Q. Gylstorff" <Quirin.Gylstorff@siemens.com> wrote:
> 
>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>>
>> Some packages even if the are part of minbase are not necessary
>> to run a debian system. Debian has some issues and experiments
>> to remove packages from minbase [1]. This feature allows a
>> expert user to remove packages from the final image during post
>> processing.
>>
>> The reason for this are e.g. disk usage reduction and reduction
>> of the clearing effort.
>>
>> The method to remove packages in postprocessing is a best-effort
>> action.
>>
>> Another way would be to identify like [1] packages in
>> minbase which can be removed without compremising the isar rootfs
>> creation and remove them directly after or during bootstrapping.
>> If a package is used for a production related feature it should be
>> reinstalled.
>>
>>
>> [1]: https://wiki.debian.org/BusterPriorityRequalification
>>
>> Quirin Gylstorff (2):
>>    meta/classes: Add remove packages to rootfs postprocessing
>>    meta-isar/images: Remove gcc-8-base from rootfs
>>
>>   meta-isar/recipes-core/images/isar-image-base.bb |  4 ++++
>>   meta/classes/image.bbclass                       |  2 +-
>>   meta/classes/rootfs.bbclass                      | 13 +++++++++++++
>>   3 files changed, 18 insertions(+), 1 deletion(-)
>>
> 

-- 
Quirin Gylstorff

Siemens AG
Corporate Technology
Research in Digitalization and Automation
Smart Embedded Systems
CT RDA IOT SES-DE
Otto-Hahn-Ring 6
81739 Muenchen, Germany
Mobile: +49 173 3746683
mailto:quirin.gylstorff@siemens.com
www.siemens.com/ingenuityforlife

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim 
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and 
Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, 
Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered offices: Berlin 
and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 
12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322

Important notice: This e-mail and any attachment thereof contain 
corporate proprietary information. If you have received it by mistake, 
please notify us immediately by reply e-mail and delete this e-mail and 
its attachments from your system. Thank you.

Re: [RFC 0/2] Remove Packages during Postprocessing

[Henning Schild]

Am Tue, 25 Feb 2020 06:43:55 +0100
schrieb Gylstorff Quirin <quirin.gylstorff@siemens.com>:

> On 2/24/20 3:24 PM, Henning Schild wrote:
> > Hi,
> > 
> > my opinion on that is clear. Fix it upstream or live with those
> > packages. You are either on a distro or fiddle around and tune
> > everything until you are the only one on the planet testing your
> > setup. That is Isar vs. yocto ... whoever thinks they _need_ that
> > should maybe think again. If they need it they can put it into
> > their own layer or use yocto ;).
> > I do not think upstream should carry such hacky features unless we
> > get better reasoning ... Removing "required" packages has the
> > potential to break your image in funny ways ... that is much more
> > expensive than a few MB disc space. All affected packages are
> > likely already cleared and vulnerabilty monitored by someone else,
> > find that someone and share the cost!
> > 
> > Henning  
> 
> Hi Henning,
> 
> I understand your concern and I think you are right. But some people
> already hack the build process in similar ways and this is a way to
> give them some support.

I guess it might be a good idea to tell those hackers to comment here
or share their reasons with you. My guess is that their need is
questionable and they did not fully understand the consequences. It
should probably be discarded as premature optimization and removed from
the downstream layer, instead of added upstream.

Even if they carefully looked at the consequences for the packages they
remove, a generic upstream feature would ease the hack for people less
careful.
 
> I want to collect the option of community regarding this patch.

Sure.

Henning

> Quirin
> 
> > 
> > On Fri, 21 Feb 2020 15:53:46 +0100
> > "Q. Gylstorff" <Quirin.Gylstorff@siemens.com> wrote:
> >   
> >> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> >>
> >> Some packages even if the are part of minbase are not necessary
> >> to run a debian system. Debian has some issues and experiments
> >> to remove packages from minbase [1]. This feature allows a
> >> expert user to remove packages from the final image during post
> >> processing.
> >>
> >> The reason for this are e.g. disk usage reduction and reduction
> >> of the clearing effort.
> >>
> >> The method to remove packages in postprocessing is a best-effort
> >> action.
> >>
> >> Another way would be to identify like [1] packages in
> >> minbase which can be removed without compremising the isar rootfs
> >> creation and remove them directly after or during bootstrapping.
> >> If a package is used for a production related feature it should be
> >> reinstalled.
> >>
> >>
> >> [1]: https://wiki.debian.org/BusterPriorityRequalification
> >>
> >> Quirin Gylstorff (2):
> >>    meta/classes: Add remove packages to rootfs postprocessing
> >>    meta-isar/images: Remove gcc-8-base from rootfs
> >>
> >>   meta-isar/recipes-core/images/isar-image-base.bb |  4 ++++
> >>   meta/classes/image.bbclass                       |  2 +-
> >>   meta/classes/rootfs.bbclass                      | 13
> >> +++++++++++++ 3 files changed, 18 insertions(+), 1 deletion(-)
> >>  
> >   
> 

Re: [RFC 0/2] Remove Packages during Postprocessing

[Gylstorff Quirin]

On 2/25/20 2:10 PM, Henning Schild wrote:
> Am Tue, 25 Feb 2020 06:43:55 +0100
> schrieb Gylstorff Quirin <quirin.gylstorff@siemens.com>:
> 
>> On 2/24/20 3:24 PM, Henning Schild wrote:
>>> Hi,
>>>
>>> my opinion on that is clear. Fix it upstream or live with those
>>> packages. You are either on a distro or fiddle around and tune
>>> everything until you are the only one on the planet testing your
>>> setup. That is Isar vs. yocto ... whoever thinks they _need_ that
>>> should maybe think again. If they need it they can put it into
>>> their own layer or use yocto ;).
>>> I do not think upstream should carry such hacky features unless we
>>> get better reasoning ... Removing "required" packages has the
>>> potential to break your image in funny ways ... that is much more
>>> expensive than a few MB disc space. All affected packages are
>>> likely already cleared and vulnerabilty monitored by someone else,
>>> find that someone and share the cost!
>>>
>>> Henning
>>
>> Hi Henning,
>>
>> I understand your concern and I think you are right. But some people
>> already hack the build process in similar ways and this is a way to
>> give them some support.
> 
> I guess it might be a good idea to tell those hackers to comment here
> or share their reasons with you. My guess is that their need is
> questionable and they did not fully understand the consequences. It
> should probably be discarded as premature optimization and removed from
> the downstream layer, instead of added upstream.
> 
> Even if they carefully looked at the consequences for the packages they
> remove, a generic upstream feature would ease the hack for people less
> careful.

FYI

I got the same patch yesterday again for stripping a rescue system to
fit into a NOR Flash.

Quirin

Re: [RFC 0/2] Remove Packages during Postprocessing

[Jan Kiszka]

On 17.03.20 11:23, Gylstorff Quirin wrote:
> 
> 
> On 2/25/20 2:10 PM, Henning Schild wrote:
>> Am Tue, 25 Feb 2020 06:43:55 +0100
>> schrieb Gylstorff Quirin <quirin.gylstorff@siemens.com>:
>>
>>> On 2/24/20 3:24 PM, Henning Schild wrote:
>>>> Hi,
>>>>
>>>> my opinion on that is clear. Fix it upstream or live with those
>>>> packages. You are either on a distro or fiddle around and tune
>>>> everything until you are the only one on the planet testing your
>>>> setup. That is Isar vs. yocto ... whoever thinks they _need_ that
>>>> should maybe think again. If they need it they can put it into
>>>> their own layer or use yocto ;).
>>>> I do not think upstream should carry such hacky features unless we
>>>> get better reasoning ... Removing "required" packages has the
>>>> potential to break your image in funny ways ... that is much more
>>>> expensive than a few MB disc space. All affected packages are
>>>> likely already cleared and vulnerabilty monitored by someone else,
>>>> find that someone and share the cost!
>>>>
>>>> Henning
>>>
>>> Hi Henning,
>>>
>>> I understand your concern and I think you are right. But some people
>>> already hack the build process in similar ways and this is a way to
>>> give them some support.
>>
>> I guess it might be a good idea to tell those hackers to comment here
>> or share their reasons with you. My guess is that their need is
>> questionable and they did not fully understand the consequences. It
>> should probably be discarded as premature optimization and removed from
>> the downstream layer, instead of added upstream.
>>
>> Even if they carefully looked at the consequences for the packages they
>> remove, a generic upstream feature would ease the hack for people less
>> careful.
> 
> FYI
> 
> I got the same patch yesterday again for stripping a rescue system to
> fit into a NOR Flash.
> 

We should move on with the series, looking into technical issues (if 
any). I do think such a functionality is better provided upstream than 
hacked up multiple times downstream.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [RFC 0/2] Remove Packages during Postprocessing

[Gylstorff Quirin]

On 2/25/20 2:10 PM, Henning Schild wrote:
> Am Tue, 25 Feb 2020 06:43:55 +0100
> schrieb Gylstorff Quirin <quirin.gylstorff@siemens.com>:
> 
>> On 2/24/20 3:24 PM, Henning Schild wrote:
>>> Hi,
>>>
>>> my opinion on that is clear. Fix it upstream or live with those
>>> packages. You are either on a distro or fiddle around and tune
>>> everything until you are the only one on the planet testing your
>>> setup. That is Isar vs. yocto ... whoever thinks they _need_ that
>>> should maybe think again. If they need it they can put it into
>>> their own layer or use yocto ;).
>>> I do not think upstream should carry such hacky features unless we
>>> get better reasoning ... Removing "required" packages has the
>>> potential to break your image in funny ways ... that is much more
>>> expensive than a few MB disc space. All affected packages are
>>> likely already cleared and vulnerabilty monitored by someone else,
>>> find that someone and share the cost!
>>>
>>> Henning
>>
>> Hi Henning,
>>
>> I understand your concern and I think you are right. But some people
>> already hack the build process in similar ways and this is a way to
>> give them some support.
> 
> I guess it might be a good idea to tell those hackers to comment here
> or share their reasons with you. My guess is that their need is
> questionable and they did not fully understand the consequences. It
> should probably be discarded as premature optimization and removed from
> the downstream layer, instead of added upstream.
> 
> Even if they carefully looked at the consequences for the packages they
> remove, a generic upstream feature would ease the hack for people less
> careful.
>   

I got another patch similar to this one to create a rescue image stored 
in nor flash.

Quirin

Re: [RFC 0/2] Remove Packages during Postprocessing

[Balasubramanian Sundaram]

[-- Attachment #1: Type: text/plain, Size: 5007 bytes --]

Hi Team,

Iam using ISAR build system for imx6 nxp based processor, I need to remove 
some user binaries like perl, shared-mime-info,libx11-6,
can you please provide steps or which file i need to change to remove these 
binaries in the final .wic.image

Thanks & Regards

On Tuesday, February 25, 2020 at 11:13:57 AM UTC+5:30 Gylstorff Quirin 
wrote:

>
>
> On 2/24/20 3:24 PM, Henning Schild wrote:
> > Hi,
> > 
> > my opinion on that is clear. Fix it upstream or live with those
> > packages. You are either on a distro or fiddle around and tune
> > everything until you are the only one on the planet testing your setup.
> > That is Isar vs. yocto ... whoever thinks they _need_ that should maybe
> > think again. If they need it they can put it into their own layer or
> > use yocto ;).
> > I do not think upstream should carry such hacky features unless we get
> > better reasoning ... Removing "required" packages has the potential to
> > break your image in funny ways ... that is much more expensive than a
> > few MB disc space. All affected packages are likely already cleared and
> > vulnerabilty monitored by someone else, find that someone and share the
> > cost!
> > 
> > Henning
>
> Hi Henning,
>
> I understand your concern and I think you are right. But some people
> already hack the build process in similar ways and this is a way to give
> them some support.
>
> I want to collect the option of community regarding this patch.
>
> Quirin
>
> > 
> > On Fri, 21 Feb 2020 15:53:46 +0100
> > "Q. Gylstorff" <Quirin.G...@siemens.com> wrote:
> > 
> >> From: Quirin Gylstorff <quirin.g...@siemens.com>
> >>
> >> Some packages even if the are part of minbase are not necessary
> >> to run a debian system. Debian has some issues and experiments
> >> to remove packages from minbase [1]. This feature allows a
> >> expert user to remove packages from the final image during post
> >> processing.
> >>
> >> The reason for this are e.g. disk usage reduction and reduction
> >> of the clearing effort.
> >>
> >> The method to remove packages in postprocessing is a best-effort
> >> action.
> >>
> >> Another way would be to identify like [1] packages in
> >> minbase which can be removed without compremising the isar rootfs
> >> creation and remove them directly after or during bootstrapping.
> >> If a package is used for a production related feature it should be
> >> reinstalled.
> >>
> >>
> >> [1]: https://wiki.debian.org/BusterPriorityRequalification
> >>
> >> Quirin Gylstorff (2):
> >> meta/classes: Add remove packages to rootfs postprocessing
> >> meta-isar/images: Remove gcc-8-base from rootfs
> >>
> >> meta-isar/recipes-core/images/isar-image-base.bb | 4 ++++
> >> meta/classes/image.bbclass | 2 +-
> >> meta/classes/rootfs.bbclass | 13 +++++++++++++
> >> 3 files changed, 18 insertions(+), 1 deletion(-)
> >>
> > 
>
> -- 
> Quirin Gylstorff
>
> Siemens AG
> Corporate Technology
> Research in Digitalization and Automation
> Smart Embedded Systems
> CT RDA IOT SES-DE
> Otto-Hahn-Ring 6
> 81739 Muenchen, Germany
> Mobile: +49 173 3746683 <+49%20173%203746683>
> mailto:quirin.g...@siemens.com
> www.siemens.com/ingenuityforlife
>
> Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim 
> Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and 
> Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, 
> Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered offices: Berlin 
> and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 
> 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322
>
> Important notice: This e-mail and any attachment thereof contain 
> corporate proprietary information. If you have received it by mistake, 
> please notify us immediately by reply e-mail and delete this e-mail and 
> its attachments from your system. Thank you.
>

-- 
CONFIDENTIALITY
This e-mail message and any attachments thereto, is 
intended only for use by the addressee(s) named herein and may contain 
legally privileged and/or confidential information. If you are not the 
intended recipient of this e-mail message, you are hereby notified that any 
dissemination, distribution or copying of this e-mail message, and any 
attachments thereto, is strictly prohibited.  If you have received this 
e-mail message in error, please immediately notify the sender and 
permanently delete the original and any copies of this email and any prints 
thereof.
ABSENT AN EXPRESS STATEMENT TO THE CONTRARY HEREINABOVE, THIS 
E-MAIL IS NOT INTENDED AS A SUBSTITUTE FOR A WRITING.  Notwithstanding the 
Uniform Electronic Transactions Act or the applicability of any other law 
of similar substance and effect, absent an express statement to the 
contrary hereinabove, this e-mail message its contents, and any attachments 
hereto are not intended to represent an offer or acceptance to enter into a 
contract and are not otherwise intended to bind the sender, Sanmina 
Corporation (or any of its subsidiaries), or any other person or entity.

[-- Attachment #2: Type: text/html, Size: 6921 bytes --]

Re: [RFC 0/2] Remove Packages during Postprocessing

[Henning Schild]

Hi Sundaram,

Am Wed, 21 Sep 2022 22:55:07 -0700 (PDT)
schrieb "'Balasubramanian Sundaram' via isar-users"
<isar-users@googlegroups.com>:

> Hi Team,
> 
> Iam using ISAR build system for imx6 nxp based processor, I need to
> remove some user binaries like perl, shared-mime-info,libx11-6,
> can you please provide steps or which file i need to change to remove
> these binaries in the final .wic.image

If the binaries are in the rootfs they are coming in as
runtime dependency of something. Or maybe the packages have been
explicitly installed by accident.

Removing rdeps is risky, you never know what you might break. So you
should think _really_ _really_ carefully about the reasons to remove
things. Question those reasons several times. Weird legal or security
reasons are better resolved in other ways. Even if one "firmware" works
with forcefully removed rdeps, the next version might break in funny
ways ... or you see the breakage in the field, on a code-path never
seen during testing.

If it is about size i would advise to rather choose a board with bigger
mass storage, a product released today where the firmware barely fits
... is not going to live too long and the hacks will get worse over the
years. Maybe try some sort of filesystem compression instead.

So now that we are clear that it would be a really bad idea ...

You best bet might be to give the patches a try. And give feedback on
how well they worked for you ... or not. Should you be using kas, it is
easy to apply patches to isar and describe all that in a kas-file in
your layer.

regards,
Henning

> Thanks & Regards
> 
> On Tuesday, February 25, 2020 at 11:13:57 AM UTC+5:30 Gylstorff
> Quirin wrote:
> 
> >
> >
> > On 2/24/20 3:24 PM, Henning Schild wrote:  
> > > Hi,
> > > 
> > > my opinion on that is clear. Fix it upstream or live with those
> > > packages. You are either on a distro or fiddle around and tune
> > > everything until you are the only one on the planet testing your
> > > setup. That is Isar vs. yocto ... whoever thinks they _need_ that
> > > should maybe think again. If they need it they can put it into
> > > their own layer or use yocto ;).
> > > I do not think upstream should carry such hacky features unless
> > > we get better reasoning ... Removing "required" packages has the
> > > potential to break your image in funny ways ... that is much more
> > > expensive than a few MB disc space. All affected packages are
> > > likely already cleared and vulnerabilty monitored by someone
> > > else, find that someone and share the cost!
> > > 
> > > Henning  
> >
> > Hi Henning,
> >
> > I understand your concern and I think you are right. But some people
> > already hack the build process in similar ways and this is a way to
> > give them some support.
> >
> > I want to collect the option of community regarding this patch.
> >
> > Quirin
> >  
> > > 
> > > On Fri, 21 Feb 2020 15:53:46 +0100
> > > "Q. Gylstorff" <Quirin.G...@siemens.com> wrote:
> > >   
> > >> From: Quirin Gylstorff <quirin.g...@siemens.com>
> > >>
> > >> Some packages even if the are part of minbase are not necessary
> > >> to run a debian system. Debian has some issues and experiments
> > >> to remove packages from minbase [1]. This feature allows a
> > >> expert user to remove packages from the final image during post
> > >> processing.
> > >>
> > >> The reason for this are e.g. disk usage reduction and reduction
> > >> of the clearing effort.
> > >>
> > >> The method to remove packages in postprocessing is a best-effort
> > >> action.
> > >>
> > >> Another way would be to identify like [1] packages in
> > >> minbase which can be removed without compremising the isar rootfs
> > >> creation and remove them directly after or during bootstrapping.
> > >> If a package is used for a production related feature it should
> > >> be reinstalled.
> > >>
> > >>
> > >> [1]: https://wiki.debian.org/BusterPriorityRequalification
> > >>
> > >> Quirin Gylstorff (2):
> > >> meta/classes: Add remove packages to rootfs postprocessing
> > >> meta-isar/images: Remove gcc-8-base from rootfs
> > >>
> > >> meta-isar/recipes-core/images/isar-image-base.bb | 4 ++++
> > >> meta/classes/image.bbclass | 2 +-
> > >> meta/classes/rootfs.bbclass | 13 +++++++++++++
> > >> 3 files changed, 18 insertions(+), 1 deletion(-)
> > >>  
> > >   
> >
> > -- 
> > Quirin Gylstorff
> >
> > Siemens AG
> > Corporate Technology
> > Research in Digitalization and Automation
> > Smart Embedded Systems
> > CT RDA IOT SES-DE
> > Otto-Hahn-Ring 6
> > 81739 Muenchen, Germany
> > Mobile: +49 173 3746683 <+49%20173%203746683>
> > mailto:quirin.g...@siemens.com
> > www.siemens.com/ingenuityforlife
> >
> > Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim 
> > Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and 
> > Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, 
> > Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered offices:
> > Berlin and Munich, Germany; Commercial registries: Berlin
> > Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE
> > 23691322
> >
> > Important notice: This e-mail and any attachment thereof contain 
> > corporate proprietary information. If you have received it by
> > mistake, please notify us immediately by reply e-mail and delete
> > this e-mail and its attachments from your system. Thank you.
> >  
>