From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:08 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f58.google.com (mail-ed1-f58.google.com [209.85.208.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692756eN003879 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:06 +0200 Received: by mail-ed1-f58.google.com with SMTP id 4fb4d7f45d1cf-6aaa4535b3bsf8335539a12.3 for ; Fri, 02 Oct 2026 00:05:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924701; cv=pass; d=google.com; s=arc-20260327; b=p+qZTI8wntCsSZwIrXfcJ5JBv3IQhsxjIa8M6OlvG3+xCqFGISAbKNkDw2Gba5tXBf GFc4WAJ4vCnsTVg1R/zqIwVhr0ZOtiVMgkwj2EuLmN6tVBuZUBvuT1yfJIqoYk5mDcdm ITbH1j+s8D+8+9M0BCzRubUIB+JywswSxZOyvwBCJ/4iMnCRnTmCXeNcRkBAEEEsBM8L JM1treS78qY6qxTdCkcCaQuecvWEhqk+QW8WyL4oQhk6eEtlp1X9/LngNSYLmUYU9rQA 0iTP47GDxCRZ/Dw0DDu8/XJ3fkHV3W9NXk1iLG26pXryuwX4LtUQHMat3YDZqMK8OwWX lcGA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; fh=VktE8frPvUp+cJU/nxDLqoyH+32xObJhr+1dh0Ly358=; b=PeMnGS8xWRhOky+oNDtVhRMqKVL26bJPSnOx093x0kCicb5Y15x7LzJ1EsfXDT4HTV lGe/deYfCRDLCR2UUtyGaaNdUmHir1cNNSe/5QZV8mbvnoFX+Vcl0JJ+9aJRTWHuX5tH +l+vGMMBN9UQPYPwUcdZPqUW1uECHyTgls47VbRXeQpbtQGK+V2pVTyjTsS6Prs/cnYM VIPh6VHr4EfhOgN2+0p+JbWRtfKF2b08tVLn9z1Sq4w7wbqwDyvvPEzdbyqs6jQDssle 6bH6H6NW/u7p1N0cEYvqOKkFjuLD7TlDezkCnvVLzzGgU2em2eHTdmCCW94IVx2GBHV6 Rusw==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924701; x=1791529501; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; b=xDKnTr1NG4PIpRhHXEBLW11fuDme8pQGz33xZdIdI7MUla1rpqEm6bOyXI/GTQil+5 d+sMMHv1Q2BOcdK5c6qQCVrCSJHtBSpJCfKuFcJcrf6Xp1u8iT/o5J6GYimX+R6IBAxe 1HeKWeKBlHLMU+uHoTdvqpXhgc+l1d5soFo9PYrZA/+n0SoT8QKfFW1AeDSyz2YsNPMM ekEqBgjJBBf7yi5uz1RtWD5gEzKwNhwnecK2C8ZoaHbVf+9qfFowFlyiox2I4FuIh2oa GwwZe8quafOk0mhCuyOoXLb4/kmBdzV+GfsQlenCNJI7bqqT8bBFgmSaMbVtKgV8CemS qiHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924701; x=1791529501; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; b=K2MiDe+VlcFSQ6l0HiF24gAtMAUs0Y+nToFxbiJDt72eQXO1g3Oqo7+4zUdFKuzI1Z lmXkf/JldG7W9+NQ5AJUhuQlYBrUoaoMVRmNkkABUIZM6Dh8tdzqpdXA4+6OF1vmq2Ml PVQyecvL3E7T2xmcZff9OTUTIqBJMvS5/RKkSzE6Eh4Jep7AiGgLtb9TlF9c4QNsznPn tsFdoRRTpnerd+GkKufOjlgDKKMbUZor6LHCmL5066JZidN/bBqi4SQFj81LuUX7zPTb iXP3tpTqRc+44WfqNFlKnAVIh0UV5Da5vNffnG/ldH/e6v+7RP3GJOnqpSfiaHbnVzFa aJGg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBy+NBNs6FO7LfAEE9DEgkin4lxra7ZaHugYOpcxw6pzmGrupaPWppL9J1lPwwFt27ipZnMa@ilbers.de X-Gm-Message-State: AFq9FYJqRRHT85yOTO/U/qzI/612Frco/CfHCpdKLkQF2RnSdzvLTS79 wrYDGCXieduhWXW+2ci6IygCrQhXczB2exi9zfUHuv+63P9hJUfGL88C X-Received: by 2002:a05:6402:2790:b0:6ac:9a44:d404 with SMTP id 4fb4d7f45d1cf-6af9de99262mr1019184a12.0.1790924700514; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcIstwvxsj898iwa6OpTFLHGMs3H87dyuQk4219OYmu/g==" Received: by 2002:a05:6402:24cc:b0:6aa:448f:b889 with SMTP id 4fb4d7f45d1cf-6af9dab6cd6ls521974a12.2.-pod-prod-05-eu; Fri, 02 Oct 2026 00:04:56 -0700 (PDT) X-Received: by 2002:a05:6402:3507:b0:6a7:ea54:38f with SMTP id 4fb4d7f45d1cf-6af9e318369mr1061654a12.32.1790924695778; Fri, 02 Oct 2026 00:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924695; cv=none; d=google.com; s=arc-20260327; b=P5h+h/qeykX8lI7M/fBY1XlsE3k1iW+EG19sAPg5DOyb9ovpFhlI5Psq9l8/U/gRM/ OLnrMfbydCmcchRuEHe+NTs5Q4BFlLKOhAEXOrjwACC1Phl2/wvQgxFPTF2xxlU7Wo+r xobPcwkP6X8QqeLZrJdRyNen0zU4IgSb7dtbaL/EMTEvEm7iCYOJz5T0YXBOw0LIPvDF 87BUT7vxMRtLljWTyVgNMZj/2/QzJH66Kt4IUBAcXIxjlMPcIH/InDbsCqjvS929kk2d ZP378G1MZFDHIGEOqX2ODwkQQo88V9KSdzbUgwDP2o9T0Y1xeHXB9YiBcQTkyA4VwGHe sMUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=DPK7XNfXmH1ik4kBjtPUHNR68d7Zw/Zr6vOXQ1ei14g=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=ioejjC6g9cCgcFZ07zl/R11/Ttb20cJZB/lDDr+EG2L+na4BEnIlQdpfqQM4OFy/pu agUKNS8I7zIdhUJAJ6nhhJUCCGYpA0HDCoYRGC1WaBLHPS9LOKk1QkrNpSjMmRiHysyB szkvrYV2r2tkTePBICdCs730f1bp8RU6iGdwcgZfvJmGsnjw0BkLiHt1y8xf52w+Gt4r IFL/TC6ftIYUKZNsrP4iPCu4nWdU/95gUBdykG4XQOb5XHX7P6NkWU6CicViKPbgqyVv tmzqc7v2G9/z9xYQ+T93+wt0ibLMguGG8SAtpQy+BYK+oaq/d62vPDokTKd8rBq77oW9 aRTw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6af9e269409si29765a12.8.2026.10.02.00.04.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1h003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:54 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 02/12] meta: Add debrepo bbclass handling base-apt prefetching Date: Fri, 2 Oct 2026 09:01:05 +0200 Message-ID: <20261002070335.26526-3-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: QDhKauSF/8Hm From: Uladzimir Bely This class uses 'scripts/debrepo' python script to prefetch given packages or sources to local base-apt repository. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- RECIPE-API-CHANGELOG.md | 10 ++++ doc/user_manual.md | 2 + meta/classes/debrepo.bbclass | 89 ++++++++++++++++++++++++++++++++++++ meta/conf/bitbake.conf | 5 ++ 4 files changed, 106 insertions(+) create mode 100644 meta/classes/debrepo.bbclass diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index abb67fa8..1951df08 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -1211,3 +1211,13 @@ ROOTFS_FEATURES:remove = "clean-apt-credentials" All `ROOTFS_POSTPROCESS_COMMAND` steps are now executed within the `do_rootfs_install` task to avoid idempotency issues on partial rebuilds. Task that previously had a dependency to `do_rootfs_postprocess` shall now be changed to run after `do_rootfs_install`. + +### "Prefetch" mode for base-apt + +Originally, `base-apt` repo is created only during second build when variable +ISAR_USE_CACHED_BASE_REPO is set. The repo is populated with every package that +took part in the first build and was cached in DL_DIR. + +New ISAR_PREFETCH_BASE_APT variable changes the way `base-apt` is populated. +Packages added to the repo before running any task that need them. Separate +`deb-repo` script is used for populating base-apt repo. diff --git a/doc/user_manual.md b/doc/user_manual.md index a134903d..6db4b8eb 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -86,12 +86,14 @@ apt install \ gettext-base \ git \ python3 \ + python3-apt \ quilt \ qemu-user-static \ reprepro \ sudo \ uidmap \ unzip \ + wget \ xz-utils \ git-buildpackage \ pristine-tar \ diff --git a/meta/classes/debrepo.bbclass b/meta/classes/debrepo.bbclass new file mode 100644 index 00000000..79a19144 --- /dev/null +++ b/meta/classes/debrepo.bbclass @@ -0,0 +1,89 @@ +# This software is a part of Isar. +# Copyright (C) 2026 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +# Prefetch to base-apt repo by default +ISAR_PREFETCH_BASE_APT ??= "1" + +DEBREPO_WORKDIR ??= "${DEBREPO_TARGET_DIR}" + +debrepo_update_apt_source_list() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + + chroot_dir=${1} + apt_list=${2} + + flock -x "${REPO_BASE_DIR}/repo.lock" -c " + sudo -E chroot ${chroot_dir} /usr/bin/apt-get update \ + -o Dir::Etc::SourceList=\"sources.list.d/${apt_list}.list\" \ + -o Dir::Etc::SourceParts=\"-\" \ + -o APT::Get::List-Cleanup=\"0\" + " +} + +debrepo_add_packages() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + args="" + if [ "${1}" = "--srcmode" ]; then + args="${args} --srcmode" + shift + fi + + if [ "${1}" = "--isarapt" ]; then + args="${args} --extrarepo=${REPO_ISAR_DIR}/${DISTRO}" + shift + fi + + workdir="${1}" + args="${args} ${2}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${workdir}" \ + ${args} +} + +debrepo_parse_dscfile() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + dscfile="${1}" + args="" + + if [ "${ISAR_CROSS_COMPILE}" = "1" ]; then + build_arch=${HOST_ARCH} + if [ "${PACKAGE_ARCH}" != "${build_arch}" ]; then + args="--crossbuild \ + crossbuild-essential-${PACKAGE_ARCH}:${build_arch} \ + dose-distcheck:${build_arch} \ + libc-dev:${PACKAGE_ARCH} \ + libstdc++-dev:${PACKAGE_ARCH} \ + " + fi + fi + + args="${args} --extrarepo=${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + if [ -n "${DEB_BUILD_PROFILES}" ]; then + export DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES}" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${DEBREPO_WORKDIR}" \ + --dscfile="${dscfile}" \ + ${args} +} diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index e69ec3bb..c8503ec6 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -67,6 +67,11 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${DISTRO}-${MACHINE}" CACHE = "${TMPDIR}/cache" KERNEL_FILE ?= "${@ 'vmlinux' if d.getVar('DISTRO_ARCH') in ['mipsel', 'riscv64', 'arm64'] else 'vmlinuz'}" +# debrepo config +DEBREPO_DIR = "${TOPDIR}/debrepo" +DEBREPO_HOST_DIR = "${DEBREPO_DIR}/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}" +DEBREPO_TARGET_DIR = "${DEBREPO_DIR}/${DISTRO}-${DISTRO_ARCH}" + MACHINEOVERRIDES ?= "${MACHINE}" DISTROOVERRIDES ?= "${DISTRO}" OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}::${BASE_DISTRO}:${ISAR_CHROOT_MODE}:forcevariable" -- 2.53.0 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20261002070335.26526-3-akarpovich%40ilbers.de.