From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:40 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f191.google.com (mail-lj1-f191.google.com [209.85.208.191]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68LdUx017104 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:40 +0100 Received: by mail-lj1-f191.google.com with SMTP id 38308e7fff4ca-2fb3e48fb66sf3743161fa.2 for ; Wed, 06 Nov 2024 00:21:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881294; cv=pass; d=google.com; s=arc-20240605; b=VmQwJQbuM25qfC1/DY7Li/eKBeV9C6wEq1eEVn1w27IT04Jc4TPrCVPq6BDvSZQZFh yJD4v+FmS5T3xbXkUMzcLE8rdsN33sl7/zBevwfB3/e90UE/EVq66d8gafhHlp0+cs47 5+IjVwztwMrd0qBg03nM0E7x9wY9k14nv45I2T7O4fXqVv1Dha9UVU6PfsxiwDRactn5 yFOJ0uvzeX7zdPgsZZNNy8VJmTmKF9O1dbVp3Yz/Vp3sMmNa6emSf6VktlqQfEKXCn+3 TkdqW/RFXARgb3YKtK5bLI+JHxp4unDL1nkZnbkFtpkBcz8AFovKJOFOzcaczqD1f55I 5H0A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=fNbPD93Ni6IXi4nxpMBsz/N/ABtxqebG0Tfp1ktjE/M=; fh=C7Y8QQAzyvrfxgX7T8Fczl7ZmaHyo7PmpmAeZRzn22Y=; b=B7UI+qAt+XNKaDFT9hsb/3VAksArSd+WGKEj4tbblZI7PUrq7mAb5Aod3HO5hFrIZw IrzJafYtlvdXagLPiOn2qgze82e7wWDcc/WGmvEB3qlyBnzCInwoovU87iVdHSADmlTZ CfnQbQyLezuKoZfQFb1UHMKMCwbwLynZQlHzUSsy2hspoYelU5ZYcuZYu+2QfPOkl29o Nc1y1g+ec3aPMZ08nW8aFcfsdJcjjRhLD/tDucWdTrrzfa0v/qOWy6S5So5eM+du74mF 0fYbHrd/iV+vlPn8sX42Y27ktPeLrTktCiUYc9LOQddDwHhRnSKCKiqfuNayGKAyqqvu rgDQ==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881294; x=1731486094; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=fNbPD93Ni6IXi4nxpMBsz/N/ABtxqebG0Tfp1ktjE/M=; b=VVQKLHLhUY9R0wU5966puvflYNySs0Dy6Fdut9gewEW6+aq2q+rJFwMXr592+9Cn/R 93y0ce60YCxFWme5OaqLIA6kF7LvidnRBW4/GgKTCx5DIb/5FDLfoEAtm5JzQhcXw00p tG5+trCwajtvZE86kPFnr2JQuYLj136NmVQEpdu1DnMaDiippR0afbmt8UJMGYrknn4T 1YUE9E5oTiXXrwUo+7wqxWcEewdCi9lIQD2yOLEHQt6UGgvXCHZL1zMnoLAoLibPDkyJ enpaKoUrw5wPPX+S8X3h0qFWdtYSRYRBgZGmbL0aRXZ6nyoQ13OIcFFINwz7+SWB5FWO 4r3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881294; x=1731486094; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=fNbPD93Ni6IXi4nxpMBsz/N/ABtxqebG0Tfp1ktjE/M=; b=kaQ3VoRbouaMJS43ld/VQ2rTsooDKIysqw9UVImVnnQGY1v4+UOrQ9EZlW7FT8Q689 PJ/HctH66sG+t/khYDM/97MjeTlyAFVFVU/Pg5rotOriCiiNKHjDGTiKVEM7rllmnzy/ BB/AbsTycfhrCVxWttLL4gIaa8UkXYJlGb3AeibGDovx5k4IuDXWN86hBt28bmw9PowT cRQFpenmr7wF2Rl0dl6rcX2uYaNsc/novHHeeayhrCJ4+/UryNqS5obgN9OvtbskCZiF H0RhnCIH2F6r0nXkIumdOEy9SBrROw+i3FgNejUG0ajrvijxKokfnckxYO4BEqgszR73 lQPA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCV8wIqxYLiQiuxxqpmoKrAzM7+6WWqXlla4Sye8f7rR3Y0es+U/3rgYKfPANQhc0TUV4wvc@ilbers.de X-Gm-Message-State: AOJu0Yx4/FlXbmYyJMa88M8ZSjym5yW0AifVoNItkhLk9n8T7b/Umew6 zuNd9TuSi8MkMUwHvNpBb6KLYtPSsmw474+L9AQ7DYNjzMyrkVHp X-Google-Smtp-Source: AGHT+IEjsq1CstabVEN64wMMpTbEpFgDkxkyoBbfSMvcBx4jphXoFqMkAYjgAqgmArfSAAtbW7S6Cw== X-Received: by 2002:a2e:be85:0:b0:2fb:56c1:bf with SMTP id 38308e7fff4ca-2fcbe04bb4dmr61645511fa.7.1730881293473; Wed, 06 Nov 2024 00:21:33 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:1f01:0:b0:2fb:358c:f74d with SMTP id 38308e7fff4ca-2fdeb667f6als13144761fa.2.-pod-prod-04-eu; Wed, 06 Nov 2024 00:21:31 -0800 (PST) X-Received: by 2002:a2e:b8c5:0:b0:2fc:9afe:1157 with SMTP id 38308e7fff4ca-2fedb794dc0mr87374451fa.2.1730881291091; Wed, 06 Nov 2024 00:21:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881291; cv=none; d=google.com; s=arc-20240605; b=Vldtgr24Fn9c8yJTA0niovUVIXpqRhvzEroZ3jAweNWkpZiDoHFvsEX1qfbn/oAdhf hFZpVVdSzyO/1ZhUUfZkxFZ83HeORDZs2c0BOyZFykNPNLn705oKWZSpTK1YQNmKw+Hn HKnxeyiQqLPimobc1mwp8e+QSqR9wjlAuzTT4iT72yLdV9Ttlx/qXzgUujal6fKbIiDz zYgEOJyokxvzoVCvVQZled3yuXAAA15k7t+f8r4UDR4aOXKCSc1mvpy/aTMCEUbPaN4D oiUWJk1/OG5qlWMcQgJ0DnjYD45jNKuZaQO4vV7gfnwzZ3JeOWhxiodvEMMjEGvTPnBs gcAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=FTzM+SpSU14IEU9VKNUCKCG3FornmhDhsewSCHJ+vT8=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=FLXGTupw0m7Pv7IbWcrWpDEHbD9D6GJNJfsjsxeDi9XvjhD+GathRlWt7djULV+wUN U9bMp897i+jqf97dPR6RB7tPhNqIaTCv94bxj4Z4+M0hcaWn0MuUZ/Z8dRuKhwv1qubW HKwsuH6ZwDNI5l9ZYfcDFtxHqfVQpeQBFlnOSEcKF/ydmquV0pEMSMl7k68ALcZThDV8 LWwBm96BxDQ1e0gP4Zp3UnK0Ea3JRYVeQhSuWhb3hiPEUzhZPIn6c2fmVl1ngpI8KRTf 2YBW6jV42fT1kAs/HbfEDItYmFoykUBYhVwXqE7OWcOTXn0Y2cChqAj1rQ+eKviYQFQx 9Tsg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-5cee6b05269si128500a12.3.2024.11.06.00.21.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:31 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjw016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:30 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 7/8] isar-bootstrap: Use tar output instead of directory Date: Wed, 6 Nov 2024 10:21:16 +0200 Message-Id: <20241106082117.1089554-8-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: quw5xm6dmjn+ Do not unpack tar with rootfs content inside bootstrap recipe. This change can allow to remove sudo usage during the bootstrap later. Using tar output requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich --- meta/classes/rootfs.bbclass | 11 +++++--- .../isar-bootstrap/isar-bootstrap.inc | 25 ++++++++++--------- .../isar-mmdebstrap/isar-mmdebstrap.inc | 22 ++++------------ 3 files changed, 25 insertions(+), 33 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index e339d24f..dee724a0 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -105,8 +105,8 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" def rootfs_extra_import(d): bb.utils._context["rootfs_progress"] = __import__("rootfs_progress") @@ -116,7 +116,10 @@ ROOTFS_EXTRA_IMPORTED := "${@rootfs_extra_import(d)}" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -275,7 +278,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 0e5ab57d..f9dae26a 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -183,9 +183,8 @@ do_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y dpkg - # setup chroot + # prepare setup chroot script install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ @@ -199,8 +198,12 @@ do_bootstrap() { mountpoint -q "${ROOTFSDIR}/base-apt" && \ umount "${ROOTFSDIR}/base-apt" + # Compress rootfs for compatibility + lopts="--one-file-system --exclude=var/cache/apt/archives" + tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" $lopts -C "${ROOTFSDIR}" . + # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -216,19 +219,17 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 4128ad80..1043f2d1 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -11,7 +11,6 @@ inherit deb-dl-dir FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" -ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" @@ -73,9 +72,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -182,17 +178,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize bootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize bootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -206,7 +194,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } @@ -214,8 +202,8 @@ bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR # we should restore symlinks after using tar if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } -- 2.34.1 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20241106082117.1089554-8-amikan%40ilbers.de.