* [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess [not found] <vekdXs12UKQ> @ 2025-03-22 18:30 ` 'Badrikesh Prusty' via isar-users 2025-04-01 12:43 ` Uladzimir Bely 0 siblings, 1 reply; 2+ messages in thread From: 'Badrikesh Prusty' via isar-users @ 2025-03-22 18:30 UTC (permalink / raw) To: isar-users; +Cc: badrikesh prusty From: badrikesh prusty <badrikesh.prusty@siemens.com> When software is compiled, it's often "stripped" of debugging symbols to reduce its size. Debug packages contain these debugging symbols. This allows developers to get detailed stack traces when a program crashes. This helps developers identify the exact location and cause of the crash. Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are cached during rootfs_postprocess. Caching is disabled by default. To enable it, set BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build, the rootfs_postprocess task downloads these packages to the rootfs, and all the dbg packages syncing with the DEBDIR with the help of rootfs_export_package_cache function. A local.conf.sample option is added to enable this feature, allowing users to uncomment and activate caching as needed. Signed-off-by: badrikesh prusty <badrikesh.prusty@siemens.com> --- meta-isar/conf/local.conf.sample | 3 +++ meta/classes/deb-dl-dir.bbclass | 18 ++++++++++++++++++ meta/classes/image.bbclass | 1 + meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++ 4 files changed, 44 insertions(+) diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index b59f310b..52ea2fb5 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -172,6 +172,9 @@ IMAGE_INSTALL:append:qemuamd64-sb = " example-module-signed-${KERNEL_NAME}" # Without this feature, only sources of packages downloaded with apt:// are downloaded. #BASE_REPO_FEATURES ?= "cache-deb-src" +# Uncomment this to enable caching of all debug symbol packages. +#BASE_REPO_FEATURES += "cache-dbg-pkgs" + # # Uncomment this to enable use of cached base repository #ISAR_USE_CACHED_BASE_REPO ?= "1" diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 7ebd057e..1e3f9a16 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -70,6 +70,24 @@ debsrc_download() { debsrc_undo_mounts "${rootfs}" } +dbg_pkgs_download() { + export rootfs="$1" + + apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \ + -a "${DISTRO_ARCH}" packages \ + "${rootfs}/var/cache/apt/archives" \ + | awk '/^Package:/ {print $2}' \ + | sort -u \ + | while read pkg; do + apt-cache -o Dir=${rootfs} showsrc ${pkg} \ + | awk '/^Package-List:/,/^$/' \ + | grep -E "${pkg}-(dbg|dbgsym)" \ + | grep "${DISTRO_ARCH}" \ + | awk '!/Binary:/ {print $1}' \ + | sort -u + done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install "$@"' -- +} + deb_dl_dir_import() { export pc="${DEBDIR}/${2}" export rootfs="${1}" diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 56eca202..24c1cb38 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}" ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} " +ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs', 'cache_dbg_pkgs', '', d)} " inherit rootfs inherit sdk diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 2348e269..406e9192 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -293,6 +293,28 @@ cache_deb_src() { fi } +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}" +cache_dbg_pkgs() { + if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] || + [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then + sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar + fi + rootfs_install_resolvconf + # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and + # relies on that through out the build. Copy that state information instead of apt-get update + # which generates a new state from upstream. + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" + + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + dbg_pkgs_download ${ROOTFSDIR} + + sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf + if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] || + [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then + sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf + fi +} + ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}" rootfs_postprocess_clean_package_cache() { sudo -E chroot '${ROOTFSDIR}' \ -- 2.39.5 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250322183030.1022828-1-badrikesh.prusty%40siemens.com. ^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess 2025-03-22 18:30 ` [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess 'Badrikesh Prusty' via isar-users @ 2025-04-01 12:43 ` Uladzimir Bely 0 siblings, 0 replies; 2+ messages in thread From: Uladzimir Bely @ 2025-04-01 12:43 UTC (permalink / raw) To: Badrikesh Prusty, isar-users On Sun, 2025-03-23 at 00:00 +0530, 'Badrikesh Prusty' via isar-users wrote: > From: badrikesh prusty <badrikesh.prusty@siemens.com> > > When software is compiled, it's often "stripped" of debugging symbols > to > reduce its size. Debug packages contain these debugging symbols. This > allows > developers to get detailed stack traces when a program crashes. This > helps > developers identify the exact location and cause of the crash. > > Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are > cached > during rootfs_postprocess. Caching is disabled by default. To enable > it, set > BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build, > the > rootfs_postprocess task downloads these packages to the rootfs, and > all the dbg packages syncing with the DEBDIR with the help of > rootfs_export_package_cache function. > > A local.conf.sample option is added to enable this feature, allowing > users > to uncomment and activate caching as needed. > > Signed-off-by: badrikesh prusty <badrikesh.prusty@siemens.com> > --- > meta-isar/conf/local.conf.sample | 3 +++ > meta/classes/deb-dl-dir.bbclass | 18 ++++++++++++++++++ > meta/classes/image.bbclass | 1 + > meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++ > 4 files changed, 44 insertions(+) > > diff --git a/meta-isar/conf/local.conf.sample b/meta- > isar/conf/local.conf.sample > index b59f310b..52ea2fb5 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -172,6 +172,9 @@ IMAGE_INSTALL:append:qemuamd64-sb = " example- > module-signed-${KERNEL_NAME}" > # Without this feature, only sources of packages downloaded with > apt:// are downloaded. > #BASE_REPO_FEATURES ?= "cache-deb-src" > > +# Uncomment this to enable caching of all debug symbol packages. > +#BASE_REPO_FEATURES += "cache-dbg-pkgs" > + > # > # Uncomment this to enable use of cached base repository > #ISAR_USE_CACHED_BASE_REPO ?= "1" > diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl- > dir.bbclass > index 7ebd057e..1e3f9a16 100644 > --- a/meta/classes/deb-dl-dir.bbclass > +++ b/meta/classes/deb-dl-dir.bbclass > @@ -70,6 +70,24 @@ debsrc_download() { > debsrc_undo_mounts "${rootfs}" > } > > +dbg_pkgs_download() { > + export rootfs="$1" > + > + apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \ > + -a "${DISTRO_ARCH}" packages \ > + "${rootfs}/var/cache/apt/archives" \ > + | awk '/^Package:/ {print $2}' \ > + | sort -u \ > + | while read pkg; do > + apt-cache -o Dir=${rootfs} showsrc ${pkg} \ > + | awk '/^Package-List:/,/^$/' \ > + | grep -E "${pkg}-(dbg|dbgsym)" \ > + | grep "${DISTRO_ARCH}" \ > + | awk '!/Binary:/ {print $1}' \ > + | sort -u > + done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get > -y --download-only install "$@"' -- > +} > + > deb_dl_dir_import() { > export pc="${DEBDIR}/${2}" > export rootfs="${1}" > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 56eca202..24c1cb38 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= > "${DEPLOY_DIR_IMAGE}" > ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}" > > ROOTFS_POSTPROCESS_COMMAND:prepend = > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', > 'cache_deb_src', '', d)} " > +ROOTFS_POSTPROCESS_COMMAND:prepend = > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs', > 'cache_dbg_pkgs', '', d)} " > > inherit rootfs > inherit sdk > diff --git a/meta/classes/rootfs.bbclass > b/meta/classes/rootfs.bbclass > index 2348e269..406e9192 100644 > --- a/meta/classes/rootfs.bbclass > +++ b/meta/classes/rootfs.bbclass > @@ -293,6 +293,28 @@ cache_deb_src() { > fi > } > > +ROOTFS_POSTPROCESS_COMMAND += > "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-dbg-pkgs', > 'rootfs_export_package_cache', '', d)}" > +cache_dbg_pkgs() { > + if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] || > + [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then > + sudo mv "${ROOTFSDIR}"/etc/resolv.conf > "${ROOTFSDIR}"/etc/resolv.conf.isar > + fi > + rootfs_install_resolvconf > + # Note: ISAR updates the apt state information(apt-get update) > only once during bootstrap and > + # relies on that through out the build. Copy that state > information instead of apt-get update > + # which generates a new state from upstream. > + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top- > level="${ROOTFSDIR}" > + > + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}- > ${BASE_DISTRO_CODENAME} > + dbg_pkgs_download ${ROOTFSDIR} > + > + sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf > + if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] || > + [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then > + sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar > "${ROOTFSDIR}"/etc/resolv.conf > + fi > +} > + > ROOTFS_POSTPROCESS_COMMAND += > "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', > 'rootfs_postprocess_clean_package_cache', '', d)}" > rootfs_postprocess_clean_package_cache() { > sudo -E chroot '${ROOTFSDIR}' \ > -- > 2.39.5 Applied to next, thanks. -- Best regards, Uladzimir. -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/e8e3ca2132fab8e8b839f74fbc1cf6fe790ec543.camel%40ilbers.de. ^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2025-04-01 12:43 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <vekdXs12UKQ>
2025-03-22 18:30 ` [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess 'Badrikesh Prusty' via isar-users
2025-04-01 12:43 ` Uladzimir Bely
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox