From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Sat, 22 Mar 2025 15:25:28 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-qv1-f56.google.com (mail-qv1-f56.google.com [209.85.219.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 52MEPRBx013544 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 22 Mar 2025 15:25:27 +0100 Received: by mail-qv1-f56.google.com with SMTP id 6a1803df08f44-6e91d8a7183sf53273716d6.0 for ; Sat, 22 Mar 2025 07:25:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1742653521; x=1743258321; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:sender:from :to:cc:subject:date:message-id:reply-to; bh=CKZtqnFVNAMPRDvyuiltDxlNWR8VeBrrhxEsmnwPuNs=; b=QEsU7IpZppH22eoWaItW+K915F+CO+s3i/3Wenc4V59EZWXJM+VwBlWwN8nV9+CZjz /281jN2j4v3X1IU2+yHYm6CZqA6u1hhUBQ9AYQKleOmBlm48r6fsP/7SfnMbw8/LGKRT XCkeTa5r6Vfn6Zp/9LLBghVo3YShfvt/Y4eRXA+Jpq7sdfN6jYaPhEYhoMOgKwIXmRaa CaynkFtrNFouX7DOtdHkT9VGwv7fNZ5QZ4xX9AMr5Lwj+slYMrr2/QzzKSOIlb7sjyIP cvxycoU99f6MK9E3mLf/v9B1fV/QjTMhevCtE+UoELuIik8M9Oc4PyWA4Lpf8ACw4uK3 7qlg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1742653521; x=1743258321; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=CKZtqnFVNAMPRDvyuiltDxlNWR8VeBrrhxEsmnwPuNs=; b=PpttGg6uHxHPyfIfF/ZHi+SRn3WRDg3oybL00Ews1BV0984wWRzQh5sS3oNZ50hC47 u/iICY1+okW8ZDw7n3Z+d0oBcKPDpRV1e4xl9V7N0IG9tPBOX6M9aXW2PY2BQfe1GDnP MPVUwAMF1Ogce4EfTkYFTlkgltrlPsXg8983uHLNzvzserrHb1T/povo8K8xAcwilAm/ nmO80TcY/H26jbftED84lAfQ1zZ9krg8jeDPO/8JeBExHbI/+EVqc7SmtheXtqRzhbdp JLP7Cny4/54Fojqe3zBgHYNq+fLhSkJXPteuXTHe+341a7D0rI/xNftSwWo1vM13FOR/ 1iVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742653521; x=1743258321; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-sender:mime-version:subject:references:in-reply-to :message-id:to:from:date:x-beenthere:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=CKZtqnFVNAMPRDvyuiltDxlNWR8VeBrrhxEsmnwPuNs=; b=h8k6rcfrxcDQPLNfi1JPXxeVuO3ptqJSDaNM3M206xxHyn3dGXUM+QV9uq6qx1A7aq Oo5Y7TtgJSUaUJCMA7TcASr1GvbWUGZv8QWL9ONX8ppzkOtzh5HhuFnKq64aEw0iPO4c 2q9cqTT5fE2HGUM/dPqdxYV5EDWjXjlKmLYRVopCbNGiWAQGgKen2RYKkFdvyolvoFrV rnW0p8w3QWCh8DQ5x+bg+qJHJN8fsGd3O5ipKSq+M+tFFks/aJCVa30p2zHKrHnhn6Gi akuWLXPuHly4jfAZEjdFlgRS+4eYfmJZwuUeXpitwlyig9CXTnalUJZ7MSXQiCukPggX LWZg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=1; AJvYcCUwQv86oeMFSr9Jp5ec+OKf2nvGbW8UFfcuGcSaOnnHegJ2Z4MvbfCniwiNatIO1hZ/u5eo@ilbers.de X-Gm-Message-State: AOJu0YyOurQI4TB6M+bXYKsnXHGrUqT5bG9vZ24F8JSl3uBcEmTVaFt8 As3XOps6o+1iL+b8ppDfMDI9L+17q3y1x1PVptu/e3YqIQq5dWyV X-Google-Smtp-Source: AGHT+IGxTFwPpQQDg+4ywOL73k6eTpT8EbDjRsD8mJrb/nWb78DtWZIJgnq7DCPfc7ycuBPOKMFuqg== X-Received: by 2002:a05:6214:d6b:b0:6e8:f949:38c6 with SMTP id 6a1803df08f44-6eb3f3302b0mr122986286d6.33.1742653521106; Sat, 22 Mar 2025 07:25:21 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=ARLLPAIIj/SAXeUfL34dwsZDMU6zleC3lRYLMz0VUQhAhuYROw== Received: by 2002:ad4:5382:0:b0:6e8:efc0:7a3f with SMTP id 6a1803df08f44-6eb34b2e10dls17136496d6.2.-pod-prod-03-us; Sat, 22 Mar 2025 07:25:20 -0700 (PDT) X-Received: by 2002:a05:620a:24cf:b0:7c5:54c9:3b94 with SMTP id af79cd13be357-7c5ba1954bdmr914183585a.28.1742653519810; Sat, 22 Mar 2025 07:25:19 -0700 (PDT) Date: Sat, 22 Mar 2025 07:25:18 -0700 (PDT) From: Srinuvasan Arjunan To: isar-users Message-Id: In-Reply-To: <20250322134349.1136193-1-badrikesh.prusty@siemens.com> References: <20250322134349.1136193-1-badrikesh.prusty@siemens.com> Subject: Re: [PATCH] meta: cache dbg/dbgsym packages during rootfs_postprocess MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_155672_425213479.1742653518853" X-Original-Sender: srinuvasanasv@gmail.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: CzDsF2VI+w4F ------=_Part_155672_425213479.1742653518853 Content-Type: multipart/alternative; boundary="----=_Part_155673_2084214914.1742653518853" ------=_Part_155673_2084214914.1742653518853 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Saturday, March 22, 2025 at 7:14:31=E2=80=AFPM UTC+5:30 Badrikesh Prusty= wrote: From: badrikesh prusty =20 When software is compiled, it's often "stripped" of debugging symbols to=20 reduce its size. Debug packages contain these debugging symbols. This=20 allows=20 developers to get detailed stack traces when a program crashes. This helps= =20 developers identify the exact location and cause of the crash.=20 Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are cached= =20 during rootfs_postprocess. Caching is disabled by default. To enable it,=20 set=20 BASE_REPO_FEATURES:append =3D " cache-dbg-pkgs". During image build, the=20 rootfs_postprocess task downloads these packages to the rootfs, and=20 all the dbg packages syncing with the DEBDIR with the help of=20 rootfs_export_package_cache function.=20 Signed-off-by: badrikesh prusty =20 ---=20 meta/classes/deb-dl-dir.bbclass | 19 +++++++++++++++++++=20 meta/classes/image.bbclass | 1 +=20 meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++=20 3 files changed, 42 insertions(+)=20 diff --git a/meta/classes/deb-dl-dir.bbclass=20 b/meta/classes/deb-dl-dir.bbclass=20 index 7ebd057e..311386ed 100644=20 --- a/meta/classes/deb-dl-dir.bbclass=20 +++ b/meta/classes/deb-dl-dir.bbclass=20 @@ -70,6 +70,25 @@ debsrc_download() {=20 debsrc_undo_mounts "${rootfs}"=20 }=20 +dbg_pkgs_download() {=20 + export rootfs=3D"$1"=20 + export rootfs_distro=3D"$2"=20 +=20 + apt-ftparchive --md5=3Dno --sha1=3Dno --sha256=3Dno --sha512=3Dno \=20 + -a "${DISTRO_ARCH}" packages \=20 + "${rootfs}/var/cache/apt/archives" \=20 + | awk '/^Package:/ {print $2}' \=20 + | sort -u \=20 + | while read pkg; do=20 + apt-cache -o Dir=3D${rootfs} showsrc ${pkg} \=20 + | awk '/^Package-List:/,/^$/' \=20 + | grep -E "${pkg}-(dbg|dbgsym)" \=20 + | grep "${DISTRO_ARCH}" \=20 + | awk '!/Binary:/ {print $1}' \=20 + | sort -u=20 + done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y=20 --download-only install "$@"' --=20 +}=20 +=20 deb_dl_dir_import() {=20 export pc=3D"${DEBDIR}/${2}"=20 export rootfs=3D"${1}"=20 diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass=20 index 56eca202..24c1cb38 100644=20 --- a/meta/classes/image.bbclass=20 +++ b/meta/classes/image.bbclass=20 @@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?=3D "${DEPLOY_DIR_IMAGE}"= =20 ROOTFS_PACKAGE_SUFFIX ?=3D "${PN}-${DISTRO}-${MACHINE}"=20 ROOTFS_POSTPROCESS_COMMAND:prepend =3D=20 "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',=20 'cache_deb_src', '', d)} "=20 +ROOTFS_POSTPROCESS_COMMAND:prepend =3D=20 "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs',=20 'cache_dbg_pkgs', '', d)} "=20 inherit rootfs=20 inherit sdk=20 diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass=20 index 2348e269..bdf0196b 100644=20 --- a/meta/classes/rootfs.bbclass=20 +++ b/meta/classes/rootfs.bbclass=20 @@ -293,6 +293,28 @@ cache_deb_src() {=20 fi=20 }=20 +ROOTFS_POSTPROCESS_COMMAND +=3D "${@bb.utils.contains('ROOTFS_FEATURES',= =20 'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}"=20 +cache_dbg_pkgs() {=20 + if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||=20 + [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then=20 + sudo mv "${ROOTFSDIR}"/etc/resolv.conf=20 "${ROOTFSDIR}"/etc/resolv.conf.isar=20 + fi=20 + rootfs_install_resolvconf=20 + # Note: ISAR updates the apt state information(apt-get update) only once= =20 during bootstrap and=20 + # relies on that through out the build. Copy that state information=20 instead of apt-get update=20 + # which generates a new state from upstream.=20 + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists=20 --one-top-level=3D"${ROOTFSDIR}"=20 +=20 + deb_dl_dir_import ${ROOTFSDIR}=20 ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}=20 + dbg_pkgs_download ${ROOTFSDIR}=20 ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}=20 +=20 + sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf=20 + if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||=20 + [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then=20 + sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar=20 "${ROOTFSDIR}"/etc/resolv.conf=20 + fi=20 +}=20 +=20 ROOTFS_POSTPROCESS_COMMAND +=3D "${@bb.utils.contains('ROOTFS_FEATURES',=20 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"= =20 rootfs_postprocess_clean_package_cache() {=20 sudo -E chroot '${ROOTFSDIR}' \=20 --=20 2.39.5 You could add the step how to enable this feature if user want to cache= =20 the debug packages in their APT E.x=20 : https://github.com/ilbers/isar/blob/master/meta-isar/conf/local.conf.samp= le#L173=20 (Like deb-src caching enablement) Thanks, Srinu=20 --=20 You received this message because you are subscribed to the Google Groups "= isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/= c415630d-a4c2-4055-94a8-6c24df1a0d37n%40googlegroups.com. ------=_Part_155673_2084214914.1742653518853 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Saturday, March 22, 2025 at 7:14:31= =E2=80=AFPM UTC+5:30 Badrikesh Prusty wrote:
From: badrikesh prusty <badrikes...@siemens.com>

When software is compiled, it's often "stripped" of debugging symbols= to
reduce its size. Debug packages contain these debugging symbols. This= allows
developers to get detailed stack traces when a program crashes. This = helps
developers identify the exact location and cause of the crash.

Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are = cached
during rootfs_postprocess. Caching is disabled by default. To enable = it, set
BASE_REPO_FEATURES:append =3D " cache-dbg-pkgs". During image build, = the
rootfs_postprocess task downloads these packages to the rootfs, and
all the dbg packages syncing with the DEBDIR with the help of
rootfs_export_package_cache function.

Signed-off-by: badrikesh prusty <bad= rikes...@siemens.com>
---
meta/classes/deb-dl-dir.bbclass | 19 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++
3 files changed, 42 insertions(+)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-di= r.bbclass
index 7ebd057e..311386ed 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -70,6 +70,25 @@ debsrc_download() {
debsrc_undo_mounts "${rootfs}"
}
=20
+dbg_pkgs_download() {
+ export rootfs=3D"$1"
+ export rootfs_distro=3D"$2"
+
+ apt-ftparchive --md5=3Dno --sha1=3Dno --sha256=3Dno --sha512=3Dn= o \
+ -a "${DISTRO_ARCH}" packages \
+ "${rootfs}/var/cache/apt/archives" \
+ | awk '/^Package:/ {print $2}' \
+ | sort -u \
+ | while read pkg; do
+ apt-cache -o Dir=3D${rootfs} showsrc ${pkg} \
+ | awk '/^Package-List:/,/^$/' \
+ | grep -E "${pkg}-(dbg|dbgsym)" \
+ | grep "${DISTRO_ARCH}" \
+ | awk '!/Binary:/ {print $1}' \
+ | sort -u
+ done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get= -y --download-only install "$@"' --
+}
+
deb_dl_dir_import() {
export pc=3D"${DEBDIR}/${2}"
export rootfs=3D"${1}"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 56eca202..24c1cb38 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?=3D "${DEPLOY_DIR_IMA= GE}"
ROOTFS_PACKAGE_SUFFIX ?=3D "${PN}-${DISTRO}-${MACHINE}"
=20
ROOTFS_POSTPROCESS_COMMAND:prepend =3D "${@bb.utils.contains('BASE_R= EPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "
+ROOTFS_POSTPROCESS_COMMAND:prepend =3D "${@bb.utils.contains('BASE_R= EPO_FEATURES', 'cache-dbg-pkgs', 'cache_dbg_pkgs', '', d)} "
=20
inherit rootfs
inherit sdk
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclas= s
index 2348e269..bdf0196b 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,28 @@ cache_deb_src() {
fi
}
=20
+ROOTFS_POSTPROCESS_COMMAND +=3D "${@bb.utils.contains('ROOTFS_FEATUR= ES', 'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}"
+cache_dbg_pkgs() {
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/re= solv.conf.isar
+ fi
+ rootfs_install_resolvconf
+ # Note: ISAR updates the apt state information(apt-get update) o= nly once during bootstrap and
+ # relies on that through out the build. Copy that state informat= ion instead of apt-get update
+ # which generates a new state from upstream.
+ sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-le= vel=3D"${ROOTFSDIR}"
+
+ deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DIST= RO_CODENAME}
+ dbg_pkgs_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DIST= RO_CODENAME}
+
+ sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
+ [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
+ sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/e= tc/resolv.conf
+ fi
+}
+
ROOTFS_POSTPROCESS_COMMAND +=3D "${@bb.utils.contains('ROOTFS_FEATUR= ES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d= )}"
rootfs_postprocess_clean_package_cache() {
sudo -E chroot '${ROOTFSDIR}' \
--=20
2.39.5


=C2=A0 =C2= =A0You could add the step how to enable this feature if user want to cache = the debug packages in their APT

=C2=A0 =C2=A0E.x= :=C2=A0https://github.com/ilbers/isar/blob/master/meta-isar/conf/local.con= f.sample#L173 (Like deb-src caching enablement)

= =C2=A0 =C2=A0Thanks,
=C2=A0 =C2=A0 Srinu=C2=A0

--
You received this message because you are subscribed to the Google Groups &= quot;isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-use= rs+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-use= rs/c415630d-a4c2-4055-94a8-6c24df1a0d37n%40googlegroups.com.
------=_Part_155673_2084214914.1742653518853-- ------=_Part_155672_425213479.1742653518853--