* [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess
[not found] <vekdXs12UKQ>
@ 2025-03-22 18:30 ` 'Badrikesh Prusty' via isar-users
2025-04-01 12:43 ` Uladzimir Bely
0 siblings, 1 reply; 2+ messages in thread
From: 'Badrikesh Prusty' via isar-users @ 2025-03-22 18:30 UTC (permalink / raw)
To: isar-users; +Cc: badrikesh prusty
From: badrikesh prusty <badrikesh.prusty@siemens.com>
When software is compiled, it's often "stripped" of debugging symbols to
reduce its size. Debug packages contain these debugging symbols. This allows
developers to get detailed stack traces when a program crashes. This helps
developers identify the exact location and cause of the crash.
Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are cached
during rootfs_postprocess. Caching is disabled by default. To enable it, set
BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build, the
rootfs_postprocess task downloads these packages to the rootfs, and
all the dbg packages syncing with the DEBDIR with the help of
rootfs_export_package_cache function.
A local.conf.sample option is added to enable this feature, allowing users
to uncomment and activate caching as needed.
Signed-off-by: badrikesh prusty <badrikesh.prusty@siemens.com>
---
meta-isar/conf/local.conf.sample | 3 +++
meta/classes/deb-dl-dir.bbclass | 18 ++++++++++++++++++
meta/classes/image.bbclass | 1 +
meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++
4 files changed, 44 insertions(+)
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index b59f310b..52ea2fb5 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -172,6 +172,9 @@ IMAGE_INSTALL:append:qemuamd64-sb = " example-module-signed-${KERNEL_NAME}"
# Without this feature, only sources of packages downloaded with apt:// are downloaded.
#BASE_REPO_FEATURES ?= "cache-deb-src"
+# Uncomment this to enable caching of all debug symbol packages.
+#BASE_REPO_FEATURES += "cache-dbg-pkgs"
+
#
# Uncomment this to enable use of cached base repository
#ISAR_USE_CACHED_BASE_REPO ?= "1"
diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 7ebd057e..1e3f9a16 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -70,6 +70,24 @@ debsrc_download() {
debsrc_undo_mounts "${rootfs}"
}
+dbg_pkgs_download() {
+ export rootfs="$1"
+
+ apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
+ -a "${DISTRO_ARCH}" packages \
+ "${rootfs}/var/cache/apt/archives" \
+ | awk '/^Package:/ {print $2}' \
+ | sort -u \
+ | while read pkg; do
+ apt-cache -o Dir=${rootfs} showsrc ${pkg} \
+ | awk '/^Package-List:/,/^$/' \
+ | grep -E "${pkg}-(dbg|dbgsym)" \
+ | grep "${DISTRO_ARCH}" \
+ | awk '!/Binary:/ {print $1}' \
+ | sort -u
+ done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install "$@"' --
+}
+
deb_dl_dir_import() {
export pc="${DEBDIR}/${2}"
export rootfs="${1}"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 56eca202..24c1cb38 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "
+ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs', 'cache_dbg_pkgs', '', d)} "
inherit rootfs
inherit sdk
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 2348e269..406e9192 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,28 @@ cache_deb_src() {
fi
}
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}"
+cache_dbg_pkgs() {
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
+ fi
+ rootfs_install_resolvconf
+ # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and
+ # relies on that through out the build. Copy that state information instead of apt-get update
+ # which generates a new state from upstream.
+ sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
+
+ deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+ dbg_pkgs_download ${ROOTFSDIR}
+
+ sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
+ fi
+}
+
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"
rootfs_postprocess_clean_package_cache() {
sudo -E chroot '${ROOTFSDIR}' \
--
2.39.5
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250322183030.1022828-1-badrikesh.prusty%40siemens.com.
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess
2025-03-22 18:30 ` [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess 'Badrikesh Prusty' via isar-users
@ 2025-04-01 12:43 ` Uladzimir Bely
0 siblings, 0 replies; 2+ messages in thread
From: Uladzimir Bely @ 2025-04-01 12:43 UTC (permalink / raw)
To: Badrikesh Prusty, isar-users
On Sun, 2025-03-23 at 00:00 +0530, 'Badrikesh Prusty' via isar-users
wrote:
> From: badrikesh prusty <badrikesh.prusty@siemens.com>
>
> When software is compiled, it's often "stripped" of debugging symbols
> to
> reduce its size. Debug packages contain these debugging symbols. This
> allows
> developers to get detailed stack traces when a program crashes. This
> helps
> developers identify the exact location and cause of the crash.
>
> Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are
> cached
> during rootfs_postprocess. Caching is disabled by default. To enable
> it, set
> BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build,
> the
> rootfs_postprocess task downloads these packages to the rootfs, and
> all the dbg packages syncing with the DEBDIR with the help of
> rootfs_export_package_cache function.
>
> A local.conf.sample option is added to enable this feature, allowing
> users
> to uncomment and activate caching as needed.
>
> Signed-off-by: badrikesh prusty <badrikesh.prusty@siemens.com>
> ---
> meta-isar/conf/local.conf.sample | 3 +++
> meta/classes/deb-dl-dir.bbclass | 18 ++++++++++++++++++
> meta/classes/image.bbclass | 1 +
> meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++
> 4 files changed, 44 insertions(+)
>
> diff --git a/meta-isar/conf/local.conf.sample b/meta-
> isar/conf/local.conf.sample
> index b59f310b..52ea2fb5 100644
> --- a/meta-isar/conf/local.conf.sample
> +++ b/meta-isar/conf/local.conf.sample
> @@ -172,6 +172,9 @@ IMAGE_INSTALL:append:qemuamd64-sb = " example-
> module-signed-${KERNEL_NAME}"
> # Without this feature, only sources of packages downloaded with
> apt:// are downloaded.
> #BASE_REPO_FEATURES ?= "cache-deb-src"
>
> +# Uncomment this to enable caching of all debug symbol packages.
> +#BASE_REPO_FEATURES += "cache-dbg-pkgs"
> +
> #
> # Uncomment this to enable use of cached base repository
> #ISAR_USE_CACHED_BASE_REPO ?= "1"
> diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-
> dir.bbclass
> index 7ebd057e..1e3f9a16 100644
> --- a/meta/classes/deb-dl-dir.bbclass
> +++ b/meta/classes/deb-dl-dir.bbclass
> @@ -70,6 +70,24 @@ debsrc_download() {
> debsrc_undo_mounts "${rootfs}"
> }
>
> +dbg_pkgs_download() {
> + export rootfs="$1"
> +
> + apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
> + -a "${DISTRO_ARCH}" packages \
> + "${rootfs}/var/cache/apt/archives" \
> + | awk '/^Package:/ {print $2}' \
> + | sort -u \
> + | while read pkg; do
> + apt-cache -o Dir=${rootfs} showsrc ${pkg} \
> + | awk '/^Package-List:/,/^$/' \
> + | grep -E "${pkg}-(dbg|dbgsym)" \
> + | grep "${DISTRO_ARCH}" \
> + | awk '!/Binary:/ {print $1}' \
> + | sort -u
> + done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get
> -y --download-only install "$@"' --
> +}
> +
> deb_dl_dir_import() {
> export pc="${DEBDIR}/${2}"
> export rootfs="${1}"
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 56eca202..24c1cb38 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?=
> "${DEPLOY_DIR_IMAGE}"
> ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
>
> ROOTFS_POSTPROCESS_COMMAND:prepend =
> "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> 'cache_deb_src', '', d)} "
> +ROOTFS_POSTPROCESS_COMMAND:prepend =
> "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs',
> 'cache_dbg_pkgs', '', d)} "
>
> inherit rootfs
> inherit sdk
> diff --git a/meta/classes/rootfs.bbclass
> b/meta/classes/rootfs.bbclass
> index 2348e269..406e9192 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -293,6 +293,28 @@ cache_deb_src() {
> fi
> }
>
> +ROOTFS_POSTPROCESS_COMMAND +=
> "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-dbg-pkgs',
> 'rootfs_export_package_cache', '', d)}"
> +cache_dbg_pkgs() {
> + if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> + [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> + sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> "${ROOTFSDIR}"/etc/resolv.conf.isar
> + fi
> + rootfs_install_resolvconf
> + # Note: ISAR updates the apt state information(apt-get update)
> only once during bootstrap and
> + # relies on that through out the build. Copy that state
> information instead of apt-get update
> + # which generates a new state from upstream.
> + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-
> level="${ROOTFSDIR}"
> +
> + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-
> ${BASE_DISTRO_CODENAME}
> + dbg_pkgs_download ${ROOTFSDIR}
> +
> + sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
> + if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
> + [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
> + sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar
> "${ROOTFSDIR}"/etc/resolv.conf
> + fi
> +}
> +
> ROOTFS_POSTPROCESS_COMMAND +=
> "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache',
> 'rootfs_postprocess_clean_package_cache', '', d)}"
> rootfs_postprocess_clean_package_cache() {
> sudo -E chroot '${ROOTFSDIR}' \
> --
> 2.39.5
Applied to next, thanks.
--
Best regards,
Uladzimir.
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/e8e3ca2132fab8e8b839f74fbc1cf6fe790ec543.camel%40ilbers.de.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2025-04-01 12:43 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <vekdXs12UKQ>
2025-03-22 18:30 ` [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess 'Badrikesh Prusty' via isar-users
2025-04-01 12:43 ` Uladzimir Bely
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox