* [PATCH] meta: cache dbg/dbgsym packages during rootfs_postprocess
@ 2025-03-22 13:43 'Badrikesh Prusty' via isar-users
2025-03-22 14:25 ` Srinuvasan Arjunan
0 siblings, 1 reply; 2+ messages in thread
From: 'Badrikesh Prusty' via isar-users @ 2025-03-22 13:43 UTC (permalink / raw)
To: isar-users; +Cc: badrikesh prusty
From: badrikesh prusty <badrikesh.prusty@siemens.com>
When software is compiled, it's often "stripped" of debugging symbols to
reduce its size. Debug packages contain these debugging symbols. This allows
developers to get detailed stack traces when a program crashes. This helps
developers identify the exact location and cause of the crash.
Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are cached
during rootfs_postprocess. Caching is disabled by default. To enable it, set
BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build, the
rootfs_postprocess task downloads these packages to the rootfs, and
all the dbg packages syncing with the DEBDIR with the help of
rootfs_export_package_cache function.
Signed-off-by: badrikesh prusty <badrikesh.prusty@siemens.com>
---
meta/classes/deb-dl-dir.bbclass | 19 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++
3 files changed, 42 insertions(+)
diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 7ebd057e..311386ed 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -70,6 +70,25 @@ debsrc_download() {
debsrc_undo_mounts "${rootfs}"
}
+dbg_pkgs_download() {
+ export rootfs="$1"
+ export rootfs_distro="$2"
+
+ apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
+ -a "${DISTRO_ARCH}" packages \
+ "${rootfs}/var/cache/apt/archives" \
+ | awk '/^Package:/ {print $2}' \
+ | sort -u \
+ | while read pkg; do
+ apt-cache -o Dir=${rootfs} showsrc ${pkg} \
+ | awk '/^Package-List:/,/^$/' \
+ | grep -E "${pkg}-(dbg|dbgsym)" \
+ | grep "${DISTRO_ARCH}" \
+ | awk '!/Binary:/ {print $1}' \
+ | sort -u
+ done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install "$@"' --
+}
+
deb_dl_dir_import() {
export pc="${DEBDIR}/${2}"
export rootfs="${1}"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 56eca202..24c1cb38 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "
+ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs', 'cache_dbg_pkgs', '', d)} "
inherit rootfs
inherit sdk
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 2348e269..bdf0196b 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,28 @@ cache_deb_src() {
fi
}
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}"
+cache_dbg_pkgs() {
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
+ fi
+ rootfs_install_resolvconf
+ # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and
+ # relies on that through out the build. Copy that state information instead of apt-get update
+ # which generates a new state from upstream.
+ sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
+
+ deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+ dbg_pkgs_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+
+ sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
+ fi
+}
+
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"
rootfs_postprocess_clean_package_cache() {
sudo -E chroot '${ROOTFSDIR}' \
--
2.39.5
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250322134349.1136193-1-badrikesh.prusty%40siemens.com.
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH] meta: cache dbg/dbgsym packages during rootfs_postprocess
2025-03-22 13:43 [PATCH] meta: cache dbg/dbgsym packages during rootfs_postprocess 'Badrikesh Prusty' via isar-users
@ 2025-03-22 14:25 ` Srinuvasan Arjunan
0 siblings, 0 replies; 2+ messages in thread
From: Srinuvasan Arjunan @ 2025-03-22 14:25 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 4914 bytes --]
On Saturday, March 22, 2025 at 7:14:31 PM UTC+5:30 Badrikesh Prusty wrote:
From: badrikesh prusty <badrikes...@siemens.com>
When software is compiled, it's often "stripped" of debugging symbols to
reduce its size. Debug packages contain these debugging symbols. This
allows
developers to get detailed stack traces when a program crashes. This helps
developers identify the exact location and cause of the crash.
Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are cached
during rootfs_postprocess. Caching is disabled by default. To enable it,
set
BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build, the
rootfs_postprocess task downloads these packages to the rootfs, and
all the dbg packages syncing with the DEBDIR with the help of
rootfs_export_package_cache function.
Signed-off-by: badrikesh prusty <badrikes...@siemens.com>
---
meta/classes/deb-dl-dir.bbclass | 19 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++
3 files changed, 42 insertions(+)
diff --git a/meta/classes/deb-dl-dir.bbclass
b/meta/classes/deb-dl-dir.bbclass
index 7ebd057e..311386ed 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -70,6 +70,25 @@ debsrc_download() {
debsrc_undo_mounts "${rootfs}"
}
+dbg_pkgs_download() {
+ export rootfs="$1"
+ export rootfs_distro="$2"
+
+ apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
+ -a "${DISTRO_ARCH}" packages \
+ "${rootfs}/var/cache/apt/archives" \
+ | awk '/^Package:/ {print $2}' \
+ | sort -u \
+ | while read pkg; do
+ apt-cache -o Dir=${rootfs} showsrc ${pkg} \
+ | awk '/^Package-List:/,/^$/' \
+ | grep -E "${pkg}-(dbg|dbgsym)" \
+ | grep "${DISTRO_ARCH}" \
+ | awk '!/Binary:/ {print $1}' \
+ | sort -u
+ done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y
--download-only install "$@"' --
+}
+
deb_dl_dir_import() {
export pc="${DEBDIR}/${2}"
export rootfs="${1}"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 56eca202..24c1cb38 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
ROOTFS_POSTPROCESS_COMMAND:prepend =
"${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
'cache_deb_src', '', d)} "
+ROOTFS_POSTPROCESS_COMMAND:prepend =
"${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs',
'cache_dbg_pkgs', '', d)} "
inherit rootfs
inherit sdk
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 2348e269..bdf0196b 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,28 @@ cache_deb_src() {
fi
}
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES',
'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}"
+cache_dbg_pkgs() {
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf
"${ROOTFSDIR}"/etc/resolv.conf.isar
+ fi
+ rootfs_install_resolvconf
+ # Note: ISAR updates the apt state information(apt-get update) only once
during bootstrap and
+ # relies on that through out the build. Copy that state information
instead of apt-get update
+ # which generates a new state from upstream.
+ sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists
--one-top-level="${ROOTFSDIR}"
+
+ deb_dl_dir_import ${ROOTFSDIR}
${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+ dbg_pkgs_download ${ROOTFSDIR}
${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+
+ sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar
"${ROOTFSDIR}"/etc/resolv.conf
+ fi
+}
+
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES',
'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"
rootfs_postprocess_clean_package_cache() {
sudo -E chroot '${ROOTFSDIR}' \
--
2.39.5
You could add the step how to enable this feature if user want to cache
the debug packages in their APT
E.x
: https://github.com/ilbers/isar/blob/master/meta-isar/conf/local.conf.sample#L173
(Like deb-src caching enablement)
Thanks,
Srinu
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/c415630d-a4c2-4055-94a8-6c24df1a0d37n%40googlegroups.com.
[-- Attachment #1.2: Type: text/html, Size: 6199 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2025-03-22 14:25 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-03-22 13:43 [PATCH] meta: cache dbg/dbgsym packages during rootfs_postprocess 'Badrikesh Prusty' via isar-users
2025-03-22 14:25 ` Srinuvasan Arjunan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox