public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Aliaksei Karpovich <akarpovich@ilbers.de>
To: isar-users@googlegroups.com
Cc: Uladzimir Bely <ubely@ilbers.de>,
	Aliaksei Karpovich <akarpovich@ilbers.de>
Subject: [v8 06/14] meta: Add cache-deb-src functionality in base-apt mode
Date: Mon, 27 Jul 2026 13:26:24 +0200	[thread overview]
Message-ID: <20260727112812.2255297-7-akarpovich@ilbers.de> (raw)
In-Reply-To: <20260727112812.2255297-1-akarpovich@ilbers.de>

From: Uladzimir Bely <ubely@ilbers.de>

Fill base-apt repo with source packages.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Aliaksei Karpovich <akarpovich@ilbers.de>
---
 meta/classes-recipe/deb-dl-dir.bbclass | 45 ++++++++++++++++++++++++++
 meta/classes-recipe/rootfs.bbclass     |  7 ++++
 2 files changed, 52 insertions(+)

diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 60e2fecc..39c77151 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -27,6 +27,51 @@ debsrc_source_version_filter() {
     | sort -u
 }
 
+debsrc_fill_base_apt() {
+    export rootfs="$1"
+    export rootfs_distro="$2"
+    mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}"
+
+    debsrc_do_mounts "${rootfs}"
+
+    trap 'exit 1' INT HUP QUIT TERM ALRM USR1
+    trap 'debsrc_undo_mounts "${rootfs}"' EXIT
+
+    ( flock 9
+    set -e
+    printenv | grep -q BB_VERBOSE_LOGS && set -x
+
+    # We need temporary files for our lists of source packages
+    # trap exit of this sub-shell to remove them (this script may exit abruptly
+    # since "set -e" is used)
+    avail=$(mktemp)
+    wanted=$(mktemp)
+    trap "rm -f ${avail} ${wanted}" EXIT
+
+    # List all packages known to apt
+    run_privileged chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
+        apt-cache dumpavail \
+        | debsrc_source_version_filter > ${avail}
+
+    # Use apt-ftparchive to scan all .deb files found in the download directory
+    # and get the <source> <version> pairs that we wish to download
+    apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
+                   -a "${DISTRO_ARCH}" packages \
+                   "${REPO_BASE_DIR}" \
+    | debsrc_source_version_filter > ${wanted}
+
+    # We now have two sorted lists: source packages we want and those known to
+    # apt. We will only consider source packages that may be found in both.
+    comm -12 ${wanted} ${avail} \
+    | while read src version; do
+        debrepo_add_packages --srcmode "${DEBREPO_TARGET_DIR}" "${src}=${version}"
+    done
+
+    ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
+
+    debsrc_undo_mounts "${rootfs}"
+}
+
 debsrc_download() {
     export rootfs="$1"
     export rootfs_distro="$2"
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 3ca46bb6..5a66b4ca 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -495,6 +495,13 @@ do_cache_deb_src() {
     run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
 
     deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+
+    debsrc_fill_base_apt ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
+
+    rootfs_do_mounts
+    debrepo_update_apt_source_list "${ROOTFSDIR}" "base-apt"
+    rootfs_do_umounts
+
     debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
 
     run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf
-- 
2.43.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260727112812.2255297-7-akarpovich%40ilbers.de.

  parent reply	other threads:[~2026-07-27 11:39 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-27 11:26 [v8 00/14] Improving base-apt usage Aliaksei Karpovich
2026-07-27 11:26 ` [v8 01/14] scripts: Add debrepo python script handling base-apt Aliaksei Karpovich
2026-07-27 12:36   ` 'Quirin Gylstorff' via isar-users
2026-07-29  9:03     ` Aliaksei Karpovich
2026-07-27 11:26 ` [v8 02/14] meta: Add debrepo bbclass handling base-apt prefetching Aliaksei Karpovich
2026-07-27 11:26 ` [v8 03/14] meta: Always use base-apt repo in local mode Aliaksei Karpovich
2026-07-27 11:26 ` [v8 04/14] meta: Use cached base-apt repo to bootstrap Aliaksei Karpovich
2026-07-27 11:26 ` [v8 05/14] base-apt: Predownload packages to base-apt before install Aliaksei Karpovich
2026-07-27 11:26 ` Aliaksei Karpovich [this message]
2026-07-27 11:26 ` [v8 07/14] testsuite: Set ISAR_PREFETCH_BASE_APT by default Aliaksei Karpovich
2026-07-27 11:26 ` [v8 08/14] Disable deb-dl-dir in base-apt prefetch mode Aliaksei Karpovich
2026-07-27 11:26 ` [v8 09/14] kas: Add PREFETCH_BASE_APT config entry Aliaksei Karpovich
2026-07-27 11:53   ` 'Jan Kiszka' via isar-users
2026-07-29  8:59     ` Aliaksei Karpovich
2026-07-27 11:26 ` [v8 10/14] ci_build.sh: Install python3-apt if not installed Aliaksei Karpovich
2026-07-27 11:26 ` [v8 11/14] rootfs: Fix URL parsing Aliaksei Karpovich
2026-07-27 11:45   ` 'Jan Kiszka' via isar-users
2026-07-29  8:54     ` Aliaksei Karpovich
2026-07-27 11:26 ` [v8 12/14] rootfs: Add version and arch for downloaded package Aliaksei Karpovich
2026-07-27 11:46   ` 'Jan Kiszka' via isar-users
2026-07-28  8:25   ` 'MOESSBAUER, Felix' via isar-users
2026-07-29  8:55     ` Aliaksei Karpovich
2026-07-30  6:33       ` 'Jan Kiszka' via isar-users
2026-07-30  7:09         ` 'MOESSBAUER, Felix' via isar-users
2026-07-27 11:26 ` [v8 13/14] sbom-chroot: Fix python3-apt dependency Aliaksei Karpovich
2026-07-27 11:49   ` 'Jan Kiszka' via isar-users
2026-07-27 11:26 ` [v8 14/14] debrepo: Add using override for reprepro Aliaksei Karpovich
2026-07-27 12:35   ` 'Quirin Gylstorff' via isar-users

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260727112812.2255297-7-akarpovich@ilbers.de \
    --to=akarpovich@ilbers.de \
    --cc=isar-users@googlegroups.com \
    --cc=ubely@ilbers.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox