From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7345128867348086784 X-Received: by 2002:a19:e00f:0:b0:513:8318:c6ea with SMTP id x15-20020a19e00f000000b005138318c6eamr4006021lfg.39.1710171083021; Mon, 11 Mar 2024 08:31:23 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:4016:b0:513:1cf6:5526 with SMTP id br22-20020a056512401600b005131cf65526ls88596lfb.0.-pod-prod-03-eu; Mon, 11 Mar 2024 08:31:21 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHyWksE3y5d+kxbGqMly7NQBseSrU37RLryzeXcf0Qs4IlXgzOtTLz37FJiGNvmtHh183oR X-Received: by 2002:ac2:4ac5:0:b0:513:a723:f1b4 with SMTP id m5-20020ac24ac5000000b00513a723f1b4mr2375547lfp.61.1710171081020; Mon, 11 Mar 2024 08:31:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710171081; cv=none; d=google.com; s=arc-20160816; b=necdn8LBGA3w3bO3nmiShmjEYbg02DYIROgjLS/HrQb/YM+gIz7YwhLVRY2JGvj2Ic K2uSqe1tqeUwgYu958d5kYzc3fGTAcUrh8gqGcqUptoBnlKv4J6JLXeQwiXDdSDi/27o ZItC7EfZT3sUCT6RVJ8+YkouJ0LuNxuFC4Ag3ZopNuySPjYAyImV1AH44R8w9z2QXqm2 ohJw6KoJGYG7zqADqHNYjXypdscYEPwrTB2609XG1tjiwwO/AwYGehG/kKGuzwAvtfW5 tskC1/R0PvmiR/eFnAzJSt5Wvc2WxjZ/ZxKpvEVQLnQlsQUy85C7w3G8nb+28FJnMfP3 Y93Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3z98fq8+cBK0B3yIl98OMDcXoLMgwQnLevr6iYuCT58=; fh=403AV4hwEHWB4lTYzGdOyLsp6fLjI4P6I+0rnADDAes=; b=kA41uT0AFqPlxwlIlqpco0yceT6f1U3J9XYB1KyslVo2nemVQ4NKMY1CJsAPXBtHeZ Ysp3IlARMDQhVkJhcfwKFevf4lvlDd0KToPlLEsS/l4AM9letY8IBQAHpDn2PUECzFjA Ukkmr0bcghVpuiLXGI8obv6lVVDbhF+7zJ0M8JZmMOQsLFD9BOBHEIoHkPx0tjatZJZM YuZAnqA+GBAH69DIeB+xq+nCSY2LsQVOKS9RkY8uaupdUOCPpYWc75ux5pxxzXVj44YP zneAp1bhHNnx5A+03whDa8Od/kEgfcx95aWhtSVbqttizEmUvfDL2hrzk36v6EmzTOc6 Cb3w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id b22-20020a0565120b9600b0051332138696si324784lfv.12.2024.03.11.08.31.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 11 Mar 2024 08:31:21 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from user-B660.promwad.corp ([159.148.83.123]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 42BFVHam007311 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Mar 2024 16:31:19 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Anton Mikanovich Subject: [PATCH v8 2/8] meta: Switch to mmdebstrap Date: Mon, 11 Mar 2024 17:31:05 +0200 Message-Id: <20240311153111.3772144-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240311153111.3772144-1-amikan@ilbers.de> References: <20240311153111.3772144-1-amikan@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: wtGDAxYeG5/Q From: Uladzimir Bely Instead of debootstrap, use mmdebstrap alternative. Internally, it uses apt and allows to debootstrap the distro from multiple repositories. Signed-off-by: Uladzimir Bely Signed-off-by: Anton Mikanovich --- meta-isar/conf/distro/ubuntu-common.inc | 4 -- .../isar-bootstrap/isar-bootstrap.inc | 64 ++++++++++--------- 2 files changed, 33 insertions(+), 35 deletions(-) diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc index 9d8a843b..14de379b 100644 --- a/meta-isar/conf/distro/ubuntu-common.inc +++ b/meta-isar/conf/distro/ubuntu-common.inc @@ -18,10 +18,6 @@ HOST_DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" # kernel package name is linux-image-generic in Ubuntu KERNEL_NAME = "generic" - -# that is what debootstrap_1.0.118ubuntu1 does anyways -DISTRO_DEBOOTSTRAP_SCRIPT = "/usr/share/debootstrap/scripts/gutsy" - DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= " \ diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 585a42b5..80a7e90d 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -15,7 +15,6 @@ PV = "1.0" BOOTSTRAP_FOR_HOST ?= "0" -DEBOOTSTRAP ?= "qemu-debootstrap" ROOTFSDIR = "${WORKDIR}/rootfs" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" @@ -23,7 +22,7 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DEPLOY_ISAR_BOOTSTRAP ?= "" -DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,usrmerge" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:ubuntu = ",apt" @@ -32,6 +31,8 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES:append:raspios64 = ",apt" DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO_ARCH')}" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" FILESEXTRAPATHS:append = ":${BBPATH}" inherit deb-dl-dir @@ -42,7 +43,7 @@ python () { topdir = d.getVar("TOPDIR") # The cached repo key can be both for bootstrapping and apt package - # installation afterwards. However, debootstrap will include the key into + # installation afterwards. However, bootstrap will include the key into # the rootfs automatically thus the right place is distro_bootstrap_keys. if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): @@ -270,7 +271,7 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" @@ -282,18 +283,30 @@ do_bootstrap() { bbfatal "${DISTRO_ARCH} does not have a compat arch" fi fi - debootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" + bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then - debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" - fi - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" -a -z "${BASE_REPO_KEY}" ]; then - debootstrap_args="$debootstrap_args --no-check-gpg" + bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST debootstrap_args E + export BOOTSTRAP_FOR_HOST - sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" if [ -n "${SOURCE_DATE_EPOCH}" ]; then export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" @@ -301,16 +314,8 @@ do_bootstrap() { sudo -E -s <<'EOSUDO' set -e - if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then - arch_param="--arch=${DISTRO_ARCH}" - fi - ${DEBOOTSTRAP} $debootstrap_args \ - $arch_param \ - ${@get_distro_components_argument(d)} \ - "${@get_distro_suite(d)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" # Install apt config mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" @@ -396,7 +401,7 @@ do_bootstrap() { umount -l "${ROOTFSDIR}/sys" umount -l "${ROOTFSDIR}/base-apt" || true - # Finalize debootstrap by setting the link in deploy + # Finalize bootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -413,19 +418,16 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } -- 2.34.1