From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7169469243442855936 X-Received: by 2002:a05:6902:4d3:b0:6ca:10da:acb7 with SMTP id v19-20020a05690204d300b006ca10daacb7mr38596182ybs.475.1669724465557; Tue, 29 Nov 2022 04:21:05 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a81:48cf:0:b0:393:f144:5d4d with SMTP id v198-20020a8148cf000000b00393f1445d4dls5993918ywa.5.-pod-prod-gmail; Tue, 29 Nov 2022 04:21:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf4aFBRiHUWJE/TPIgJJHmuCHHoAiatCcEsHnVd/wAm3t7aa9ZPXhxsPgNWU0hg9pI1B4NQk X-Received: by 2002:a0d:f9c5:0:b0:388:7d2:575d with SMTP id j188-20020a0df9c5000000b0038807d2575dmr43472830ywf.21.1669724464891; Tue, 29 Nov 2022 04:21:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669724464; cv=none; d=google.com; s=arc-20160816; b=mk9+h/mAwzhFQo5h6m+wMBpYhW+2PdLWNGP6c0y0aWOZFj5GqEqgft+Zrawa5/VXGK jKsFBFZmIEFmnyWh6i9DMnrq0qv6udJW5YMIlH1I+sTG7Mi7vOkrk/hrKaLtZ+aprjl8 PAXR+E7+DPGhH8qbk+NAI3m53p4zzvD9pfmxSzKliMRX+z1A8QwEE6ZdUcQxrapPjp+6 cXi849/D6YhG/sEAdzrV1wy3bFoyPFU7x7BaUJhHvNw5oX4IcILIoZrcHFX8N2K/PZNV LiQ+gDeN9/EedFMYKjTAF6l7sGxQTi8iMvrMRbgF2gM60CSQrRF70F7hx3+OXkKFqNbu l6hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=P3We6dCuwX7cq7kKuXAvBuebqQcV3Rd/T4jfTOGnIGQ=; b=Enp8M76+yDaM1Mnb85582Bb0pzk1cMdURgW93by8U0io2Yv9umwkzeBoDxUBL3X5IF 3scueirgyfBd+gjtOU+QHgywY3OFknmfR0VXG5yRsEGxaHKHOmqOPC5o1EJYfrCjXp3P gvjMTb8PseBT5AmFI/jzVznRdb+otiOGkQRMLyq7vAIOergH7rcFdhuchGObPu4aQXUI MU+C1USgi85VFmLBmb2umHkngXXBXSSfZ8B+ywGna5TIVFE5pf4tjgbW40I7hqW/UnAq YaGKggs5nVaS4RiWDVJ1AYkZMbFqyAYO78gIIh9VGgkD16Js1JlSgRMwUT3DA/pu66Hv FMAw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id bx10-20020a05690c080a00b003704845f699si837456ywb.0.2022.11.29.04.21.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 Nov 2022 04:21:02 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2ATCKwT5001019 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 29 Nov 2022 13:21:00 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v5 4/4] Normalize deb download dirs when custom DISTRO used Date: Tue, 29 Nov 2022 13:20:58 +0100 Message-Id: <20221129122058.22288-5-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221129122058.22288-1-ubely@ilbers.de> References: <20221129122058.22288-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: BF4aE0oK4l9a This changes deb download directory to the path related to BASE_DISTRO variable (instead of DISTRO previously used). In this case, if cross-compile mode used, equal host and target packages will be placed to the same download directory, if their "base" distros are equal. In case of cross-build, this saves disk space, reduces network usage and makes download directory look cleaner. Signed-off-by: Uladzimir Bely --- meta/classes/dpkg-base.bbclass | 4 ++-- meta/classes/dpkg.bbclass | 4 ++-- meta/classes/image-tools-extension.bbclass | 4 ++-- meta/classes/rootfs.bbclass | 9 +++++---- meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++-- meta/recipes-devtools/base-apt/base-apt.bb | 13 ++++++------- .../buildchroot/buildchroot-host.bb | 1 + .../sbuild-chroot/sbuild-chroot-host.bb | 1 + 8 files changed, 21 insertions(+), 19 deletions(-) diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index e4f6711c..260aa73e 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -115,7 +115,7 @@ do_apt_fetch() { for uri in "${SRC_APT}"; do schroot -d / -c ${SBUILD_CHROOT} -- \ - sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}" + sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" "${uri}" done schroot_delete_configs } @@ -147,7 +147,7 @@ do_apt_unpack() { cd ${PP} cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} dpkg-source -x "${dscfile}" "${PPS}"' \ - my_script "${DISTRO}" "${uri}" + my_script "${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" "${uri}" done schroot_delete_configs } diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index 1f55e3e7..4ac4273f 100644 --- a/meta/classes/dpkg.bbclass +++ b/meta/classes/dpkg.bbclass @@ -52,9 +52,9 @@ dpkg_runbuild() { bbwarn "Export of '${line}' detected, please migrate to templates" done - distro="${DISTRO}" + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then - distro="${HOST_DISTRO}" + distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" fi deb_dl_dir_import "${WORKDIR}/rootfs" "${distro}" diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass index ecd4365b..101704d0 100644 --- a/meta/classes/image-tools-extension.bbclass +++ b/meta/classes/image-tools-extension.bbclass @@ -19,9 +19,9 @@ do_install_imager_deps() { exit fi - distro="${DISTRO}" + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then - distro="${HOST_DISTRO}" + distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" fi buildchroot_do_mounts diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index d19ac037..786682d9 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -6,6 +6,7 @@ inherit deb-dl-dir ROOTFS_ARCH ?= "${DISTRO_ARCH}" ROOTFS_DISTRO ?= "${DISTRO}" ROOTFS_PACKAGES ?= "" +ROOTFS_BASE_DISTRO ?= "${BASE_DISTRO}" # Features of the rootfs creation: # available features are: @@ -137,7 +138,7 @@ rootfs_install_resolvconf() { ROOTFS_INSTALL_COMMAND += "rootfs_import_package_cache" rootfs_import_package_cache[weight] = "5" rootfs_import_package_cache() { - deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_DISTRO} + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} } ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" @@ -154,7 +155,7 @@ ROOTFS_INSTALL_COMMAND += "${ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT}" ROOTFS_INSTALL_COMMAND += "rootfs_export_package_cache" rootfs_export_package_cache[weight] = "5" rootfs_export_package_cache() { - deb_dl_dir_export ${ROOTFSDIR} ${ROOTFS_DISTRO} + deb_dl_dir_export ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} } ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}" @@ -219,8 +220,8 @@ cache_deb_src() { # which generates a new state from upstream. sudo cp -Trpn "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" - deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_DISTRO} - debsrc_download ${ROOTFSDIR} ${ROOTFS_DISTRO} + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] || diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 14d876aa..b9ae16cd 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -299,7 +299,7 @@ do_bootstrap() { export BOOTSTRAP_FOR_HOST debootstrap_args E sudo rm -rf --one-file-system "${ROOTFSDIR}" - deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_DISTRO}" + deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo -E -s <<'EOSUDO' set -e @@ -407,7 +407,7 @@ do_bootstrap() { # Finalize debootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_DISTRO}" + deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" } addtask bootstrap before do_build after do_generate_keyrings diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-devtools/base-apt/base-apt.bb index 05016d80..ea885fe6 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb +++ b/meta/recipes-devtools/base-apt/base-apt.bb @@ -13,10 +13,9 @@ KEYFILES ?= "" BASE_REPO_FEATURES ?= "" populate_base_apt() { - distro="${1}" - base_distro="${2}" + base_distro="${1}" - find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do + find "${DEBDIR}"/"${base_distro}-${BASE_DISTRO_CODENAME}" -name '*\.deb' | while read package; do # NOTE: due to packages stored by reprepro are not modified, we can # use search by filename to check if package is already in repo. In # addition, md5sums are compared to ensure that the package is the @@ -41,7 +40,7 @@ populate_base_apt() { "${package}" done - find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do + find "${DEBSRCDIR}"/"${base_distro}-${BASE_DISTRO_CODENAME}" -name '*\.dsc' | while read package; do repo_add_srcpackage "${REPO_BASE_DIR}"/"${base_distro}" \ "${REPO_BASE_DB_DIR}"/"${base_distro}" \ "${BASE_DISTRO_CODENAME}" \ @@ -58,17 +57,17 @@ repo() { "${BASE_DISTRO_CODENAME}" \ "${WORKDIR}/distributions.in" \ "${KEYFILES}" - populate_base_apt "${DISTRO}" "${BASE_DISTRO}" + populate_base_apt "${BASE_DISTRO}" repo_sanity_test "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" - if [ '${DISTRO}' != '${HOST_DISTRO}' ]; then + if [ '${BASE_DISTRO}' != '${HOST_BASE_DISTRO}' ]; then repo_create "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ "${REPO_BASE_DB_DIR}"/"${HOST_BASE_DISTRO}" \ "${BASE_DISTRO_CODENAME}" \ "${WORKDIR}/distributions.in" \ "${KEYFILES}" - populate_base_apt "${HOST_DISTRO}" "${HOST_BASE_DISTRO}" + populate_base_apt "${HOST_BASE_DISTRO}" repo_sanity_test "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ "${REPO_BASE_DB_DIR}"/"${HOST_BASE_DISTRO}" fi diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index d8498a56..529578d6 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -10,6 +10,7 @@ BUILDCHROOT_VARIANT = "host" require buildchroot.inc ROOTFS_ARCH = "${HOST_ARCH}" ROOTFS_DISTRO = "${HOST_DISTRO}" +ROOTFS_BASE_DISTRO = "${HOST_BASE_DISTRO}" BUILDCHROOT_PREINSTALL ?= " \ ${BUILDCHROOT_PREINSTALL_COMMON} \ diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb index aa82846b..a37e49b7 100644 --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb @@ -11,3 +11,4 @@ require sbuild-chroot.inc ROOTFS_ARCH = "${HOST_ARCH}" ROOTFS_DISTRO = "${HOST_DISTRO}" +ROOTFS_BASE_DISTRO = "${HOST_BASE_DISTRO}" -- 2.20.1