[PATCH 1/1] rootfs postprocess: clean python cache

[PATCH 1/1] rootfs postprocess: clean python cache

[Felix Moessbauer]

When calling python scripts, python automatically creates cache files
to speedup future invocations of the same sources. This often happens
in postinst scripts, that directly run in the image chroot. The
created debian packages do not ship these files, as the debheper
scripts remove them before installing.

For the rootfs part, we manually have to do it to also not
include these in the final image. This patch implements this logic in
a custom cleanup postprocess step. As there might be situations where
shipping of a subset of the caches is desireable (e.g. readonly rootfs
images), we add support to control this logic using ROOTFS_FEATURES.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/image.bbclass  | 2 +-
 meta/classes/rootfs.bbclass | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 629a0c1..6859240 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -80,7 +80,7 @@ image_do_mounts() {
 }
 
 ROOTFSDIR = "${IMAGE_ROOTFS}"
-ROOTFS_FEATURES += "clean-package-cache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache"
+ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache"
 ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
 ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 786682d..325e7ae 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -252,6 +252,12 @@ rootfs_postprocess_clean_debconf_cache() {
     sudo rm -rf "${ROOTFSDIR}/var/cache/debconf/"*
 }
 
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-pycache', 'rootfs_postprocess_clean_pycache', '', d)}"
+rootfs_postprocess_clean_pycache() {
+    sudo find ${ROOTFSDIR}/usr -type f -name '*.pyc'       -delete -print
+    sudo find ${ROOTFSDIR}/usr -type d -name '__pycache__' -delete -print
+}
+
 ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'rootfs_generate_manifest', '', d)}"
 rootfs_generate_manifest () {
     mkdir -p ${ROOTFS_MANIFEST_DEPLOY_DIR}
-- 
2.34.1

RE: [PATCH 1/1] rootfs postprocess: clean python cache

[Venkata.Pyla]

>-----Original Message-----
>From: Felix Moessbauer <felix.moessbauer@siemens.com>
>Sent: 09 January 2023 16:59
>To: isar-users@googlegroups.com
>Cc: pyla venkata(ＴＳＩＰ TMIEC ODG Porting) <Venkata.Pyla@toshiba-
>tsip.com>; jan.kiszka@siemens.com; hayashi kazuhiro(林 和宏 □ＳＷＣ◯ＡＣ
>Ｔ) <kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(ＴＳＩＰ TMIEC ODG
>Porting) <dinesh.kumar@toshiba-tsip.com>; Felix Moessbauer
><felix.moessbauer@siemens.com>
>Subject: [PATCH 1/1] rootfs postprocess: clean python cache
>
>When calling python scripts, python automatically creates cache files to
>speedup future invocations of the same sources. This often happens in postinst
>scripts, that directly run in the image chroot. The created debian packages do
>not ship these files, as the debheper scripts remove them before installing.
>
>For the rootfs part, we manually have to do it to also not include these in the
>final image. This patch implements this logic in a custom cleanup postprocess
>step. As there might be situations where shipping of a subset of the caches is
>desireable (e.g. readonly rootfs images), we add support to control this logic
>using ROOTFS_FEATURES.
>
>Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
>---
> meta/classes/image.bbclass  | 2 +-
> meta/classes/rootfs.bbclass | 6 ++++++
> 2 files changed, 7 insertions(+), 1 deletion(-)
>
>diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index
>629a0c1..6859240 100644
>--- a/meta/classes/image.bbclass
>+++ b/meta/classes/image.bbclass
>@@ -80,7 +80,7 @@ image_do_mounts() {
> }
>
> ROOTFSDIR = "${IMAGE_ROOTFS}"
>-ROOTFS_FEATURES += "clean-package-cache generate-manifest export-dpkg-
>status clean-log-files clean-debconf-cache"
>+ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-
>manifest export-dpkg-status clean-log-files clean-debconf-cache"
> ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
> ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
>diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index
>786682d..325e7ae 100644
>--- a/meta/classes/rootfs.bbclass
>+++ b/meta/classes/rootfs.bbclass
>@@ -252,6 +252,12 @@ rootfs_postprocess_clean_debconf_cache() {
>     sudo rm -rf "${ROOTFSDIR}/var/cache/debconf/"*
> }
>
>+ROOTFS_POSTPROCESS_COMMAND +=
>"${@bb.utils.contains('ROOTFS_FEATURES', 'clean-pycache',
>'rootfs_postprocess_clean_pycache', '', d)}"
>+rootfs_postprocess_clean_pycache() {
>+    sudo find ${ROOTFSDIR}/usr -type f -name '*.pyc'       -delete -print
>+    sudo find ${ROOTFSDIR}/usr -type d -name '__pycache__' -delete
>+-print }
>+

Thanks Felix, this patch works for me.


> ROOTFS_POSTPROCESS_COMMAND +=
>"${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest',
>'rootfs_generate_manifest', '', d)}"
> rootfs_generate_manifest () {
>     mkdir -p ${ROOTFS_MANIFEST_DEPLOY_DIR}
>--
>2.34.1