From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7202866696458076160 X-Received: by 2002:ab0:54d0:0:b0:68b:a181:563c with SMTP id q16-20020ab054d0000000b0068ba181563cmr1413529uaa.0.1677048091791; Tue, 21 Feb 2023 22:41:31 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a67:eb11:0:b0:412:6005:4f8e with SMTP id a17-20020a67eb11000000b0041260054f8els1520458vso.7.-pod-prod-gmail; Tue, 21 Feb 2023 22:41:31 -0800 (PST) X-Google-Smtp-Source: AK7set81JbOebu6InBqc0Evh6LAQsc96RT2wsGrBSvcs1WSiDlRiZtjGaNJVDsr8VCPw8r/ZHpLU X-Received: by 2002:a67:d59d:0:b0:414:f18:cdf5 with SMTP id m29-20020a67d59d000000b004140f18cdf5mr6484494vsj.0.1677048090971; Tue, 21 Feb 2023 22:41:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677048090; cv=none; d=google.com; s=arc-20160816; b=Vgo4Jcbui79eYkRztXj3tzjS+wRgLF6mMwHeg91G7QigCAiK8Pxk6NRcmXxW/LQs2N qo6ReeH5322wG+gWuOFrPrCPHy0nj8sJx9qjVTCXBxvbA9ULNvsihRVnRLvrzvqS4Obd aFfDekff+PsvMO61YZNCuScp3hmkJNNCSNiIB/xopbmG7buAXBV46WE42mW57JAWmrCW ZuBRbVFiwk8bNgnCPUA5auTsms1uPjZnTy4sHexIZ4ZeliZFLpmoxEh4gz6pscEhn+qN m8mCkvFZkSHGMxhvmsb/L1MQ9jDoxB/1IqoHdS4FuHQl6q84384r5H0PdxB8aUp4lzsP Gjqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=I7X0Jd3wl00jYuKnxpVXQ76O1gcd0KMSIXlp+8mJChs=; b=BLx86rdeQ6buXMKixdPqPm0Rd4LR4HHK20daOAEMwidf4Fg5HynlP6Qp9cMsyOP/dB UuwgA2TwRpwf0vQk8NC1V6U9xoiRzEK3Ms8eZoF33kr1Zyb4YFc6bFTSCDSMJ7anFiFQ shqJKsQmtFmWdJmK71o98WR/PaHXLP3YmUBGZRudR2Si7q8bq/HNH6/2e+WWKOiwaS8H YbVAfNEgVbwklgWyWz4zCU3IOFJ1q178m+SL1ECMX5/B6lHFbpkZ6QF4/WIMTGXyfEXE Er6To7UOiYEOM0wriLdOxHbQLUgDNGDuJubgoDcUjcI4yDGDYQup7YsXVSNkLVmtyDTP DzHg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id i25-20020a67e2d9000000b0041404daed81si417252vsm.0.2023.02.21.22.41.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 21 Feb 2023 22:41:30 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 31M6fPcA011872 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 22 Feb 2023 07:41:28 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v4 06/13] meta: Use cached base-apt repo to debootstrap Date: Wed, 22 Feb 2023 07:41:18 +0100 Message-Id: <20230222064125.22754-7-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230222064125.22754-1-ubely@ilbers.de> References: <20230222064125.22754-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 14RF/2JL9Ri/ This patch makes local base-apt repo to be created before debootstrap task. So, debootstrap is then done from it. The required packages are downloaded via python-apt and reprepro creates debian-like repository from .deb files. For debian targets host keyring is used while ubuntu/raspbian targets use keys specified by DISTRO_BOOTSTRAP_KEYS variable. The goal is have workable base-apt repo before first build completed. Signed-off-by: Uladzimir Bely --- .../isar-bootstrap/isar-bootstrap.inc | 92 ++++++++++++++++--- 1 file changed, 78 insertions(+), 14 deletions(-) diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 01668b93..4a3ef919 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -29,6 +29,8 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO_ARCH')}" + FILESEXTRAPATHS:append = ":${BBPATH}" inherit deb-dl-dir @@ -273,12 +275,69 @@ do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +inherit debrepo + +debrepo_bootstrap_prepare() { + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + return + fi + + debrepo_args="" + if [ "${BASE_DISTRO}" != "debian" ]; then + if [ "${BASE_DISTRO}" != "raspbian" ] && [ "${BASE_DISTRO}" != "raspios" ] || [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then + debrepo_args="$debrepo_args --keydir=${WORKDIR}" + fi + else + if [ "${BASE_DISTRO_CODENAME}" = "sid" ]; then + debrepo_args="$debrepo_args --keydir=${WORKDIR}" + fi + fi + #if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + debrepo_args="$debrepo_args --no-check-gpg" + #fi + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + debrepo_args="$debrepo_args --compatarch=${COMPAT_DISTRO_ARCH}" + fi + + debrepo_workdir=${DEBREPO_TARGET_DIR} + if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then + debrepo_args="$debrepo_args --crossarch=${DISTRO_ARCH}" + if [ ${HOST_ARCH} != ${DISTRO_ARCH} ]; then + debrepo_workdir=${DEBREPO_HOST_DIR} + fi + else + debrepo_args="$debrepo_args --isaraptdir=${REPO_ISAR_DIR}/${DISTRO}" + fi + mkdir -p "${DEBREPO_WORKDIR}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + + flock -x "${DEBREPO_WORKDIR}/repo.lock" -c " + ${SCRIPTSDIR}/debrepo --init \ + --workdir=\"${debrepo_workdir}\" \ + --aptsrcsfile=\"${APTSRCS_INIT}\" \ + --repodir=\"${REPO_BASE_DIR}\" \ + --repodbdir=\"${REPO_BASE_DB_DIR}\" \ + --mirror=\"${@get_distro_source(d)}\" \ + --arch=\"${BOOTSTRAP_DISTRO_ARCH}\" \ + --distro=\"${BOOTSTRAP_BASE_DISTRO}\" \ + --codename=\"${BASE_DISTRO_CODENAME}\" \ + ${debrepo_args} \ + gnupg locales usr-is-merged + " +} + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then bbfatal "${DISTRO_ARCH} does not have a compat arch" fi fi + + debrepo_bootstrap_prepare + debootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" @@ -292,19 +351,21 @@ do_bootstrap() { sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + (flock 9 sudo -E -s <<'EOSUDO' set -e if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then - arch_param="--arch=${DISTRO_ARCH}" + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH}" fi - ${DEBOOTSTRAP} $debootstrap_args \ - $arch_param \ - ${@get_distro_components_argument(d)} \ - "${@get_distro_suite(d)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} - + flock -x "${REPO_BASE_DIR}/repo.lock" -c " + ${DEBOOTSTRAP} $debootstrap_args \ + $arch_param \ + ${@get_distro_components_argument(d)} \ + ${@get_distro_suite(d)} \ + ${ROOTFSDIR} \ + file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO} \ + ${DISTRO_DEBOOTSTRAP_SCRIPT} + " # Install apt config mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" install -v -m644 "${APTPREFS}" \ @@ -373,11 +434,13 @@ do_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} fi - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ - -o APT::Update::Error-Mode=any - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes + flock -x "${REPO_BASE_DIR}/repo.lock" -c " + chroot ${ROOTFSDIR} /usr/bin/apt-get update -y \ + -o APT::Update::Error-Mode=any + chroot ${ROOTFSDIR} /usr/bin/apt-get install -y -f + chroot ${ROOTFSDIR} /usr/bin/apt-get dist-upgrade -y \ + -o Debug::pkgProblemResolver=yes + " umount -l "${ROOTFSDIR}/dev/shm" umount -l "${ROOTFSDIR}/dev/pts" @@ -389,6 +452,7 @@ do_bootstrap() { # Finalize debootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO + ) 9>"${DEBREPO_WORKDIR}/repo.lock" deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" # Cleanup apt cache -- 2.20.1