From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:06 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f187.google.com (mail-lj1-f187.google.com [209.85.208.187]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I5PY030927 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:05 +0200 Received: by mail-lj1-f187.google.com with SMTP id 38308e7fff4ca-2ee86eda4e4sf36409061fa.3 for ; Sun, 07 Jul 2024 23:18:05 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419479; cv=pass; d=google.com; s=arc-20160816; b=HXu789fa78A4ywlE/g1alxub2wQgHvQgqGsOTn+XXh7CP3NbSzURIyFQ+8D+SeLnpp 7VpAftT9XSJoktAoqHPxqoWwfPjY5qLskTtiyq4vBg5YOCtvNsYHoZ4M1Lx3GRy8hWjV MfQ85NNA6i1WUljrKPMElcoGAJYkyZHNP8oTXffrWyCg0G+o3xMkW6H7Zj6GuQnGL/ej GMYF/lEt4T/xEkMjbB3KpwnU8upBzNLMT3A4BnLGDbj1y//snhz2PsBQQwBsW2/TZYO+ TLqITrqTB2LCbU1am5iLvQ3OaUHbq5Hjqi60RxWHwDOfN0qfe6GER5ZvqJ5FYFyYu5uO 1uOg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=6KeDG8EnGr4dmaEUO0kinkyPEGDCXbH9Nk1rp98cbZA=; fh=2bQkSrfltF11TgWWu/TJfwwHJ05PpcI1ei6Bnqk6o48=; b=MqnNiPZozc2eRuV7mY8j+I0qpDFUcBZWDq1IkSzYmJUqF8DRqgpo3YaeTmKNLjT/x8 WK7+DlcdxgDC5hKZrg1MTwhTJhZyMpTkoo4mEn6PLtRXxJzthIs53tMcP6Bv/n467z+N xNq96/45f/FaHbQDkBFPnIlKNpu+gva1svNjHBL+nj5KU8fyZ7XhwCecBwbDZiSGgs4g a2mRz+tmG+oA9mz5e523i0kdfMCiZjUjcZAh5mvwa8eP0m85lf7tp4XJqIPVkvemZ2gh DevfunS/9s/t3vAQ311qMsAK1Tdf2VLldcD8/jQqaIqV4yvvQL8sxf9gEpR/RQXse7EX 8nrw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419479; x=1721024279; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=6KeDG8EnGr4dmaEUO0kinkyPEGDCXbH9Nk1rp98cbZA=; b=smFWOVy65+u5bhrwqASOUvFrzbwqaTQUWz0az+TaBzLMzMmevnhjDmW+xdtsBmTX7q 859fN/FSxSqxM2hLwwRHaGE/ONmOwVgMqXZza/jZmRDYPCzZUJxBspng3Pm3VbF5GkST 8z59oWHrA1dWSCUZ6d1NooYj1h8IMWoelLVLIcq/YAKwvwpfrnhZvweEWKM6y6jpW+2G zqJF01i3P7qARQyh9guB+G+3ixCyy5j7ytVmRLdPB4O4a5dEsgZKpGcDj0cSv6225Bn5 6JEKyfQGHEHHJD61ODrz8NQ/XuuCNhotl/FSK2W39D4ESCV3Prg++hIvnxg/QRDGiM8W y1rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419479; x=1721024279; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=6KeDG8EnGr4dmaEUO0kinkyPEGDCXbH9Nk1rp98cbZA=; b=LBr42d/LUmyJoqPE0DYV+X5Wyw9YP9Oluut6HVidohMHOTgQph5DgYq//CymqN/C4V mAGkIil9QiWuOlAcjW+XxldfJlq50OuVQIJAj2US/W9WY9Ryg611b28lvEWhPbuYdeqm 2dVccYUgy5sl+eIDXGdMYBrL1SdXYFVED/kqmWMl11Bu0Gc8nOpgTyMSVZAVRsoTpPYf ON42w6LKMPh136U/1a7V5LanqUCZpeZOTCpa17XekHRvoH8K/M+j/wcd8g+tjXctpItU YZ1WVWdoIAenm6IJTRvD5CmDrQ7ipyRilK/S7tqwkZKvrTeGzZ8IYS5LQ4LmyVQLECPV WaIA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCV9MIuapugrIEDsqo34D1yavmlkkVPXjqOI/bnjqk4Zrfq+QV38Mpvo/rZn0YqATwBJD+AFEp7idLBc0RSS2Bfqpuk= X-Gm-Message-State: AOJu0Yxr9hAK+Dd+x0D3BwP/UDHdo/DhLMGfuOLMM306JeEluF4rfRFG mlbjmT55dqgej6R8Q7tg4hxBmfVO/T96qIQ5y6MkWGu/3BSd3Z+a X-Google-Smtp-Source: AGHT+IH1cSi6pnyB0DQca0odZnarkpLruuw2hkFU6xt7XxZCB4Snxr1NyH7XzhG1e3rMOQhmVf/3xg== X-Received: by 2002:a2e:a58a:0:b0:2ee:7d37:498b with SMTP id 38308e7fff4ca-2ee8ed8cf11mr92339071fa.14.1720419478929; Sun, 07 Jul 2024 23:17:58 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:aa26:0:b0:2ee:89a7:12ac with SMTP id 38308e7fff4ca-2ee8cd96adfls18073181fa.1.-pod-prod-06-eu; Sun, 07 Jul 2024 23:17:57 -0700 (PDT) X-Received: by 2002:a05:651c:601:b0:2ee:4f22:33f9 with SMTP id 38308e7fff4ca-2ee8eda42f0mr63774631fa.24.1720419475918; Sun, 07 Jul 2024 23:17:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419475; cv=none; d=google.com; s=arc-20160816; b=ix4HviUdwA1EEZBEKxNsNpDOVRLzCbKrvcOTu6v0/xTlSIigFpi40t9EbatepSw6tA H6FvyLdaP3zpPRoruJyw/0hfqs8Dr6gDVuwv0icahguesEUF646caU4+DQrLuuaPp8ID 8JjeVHEXxlzAfvoshj+I+IXw3IRVGPW54OVZV9k0RD9PDSMUaAxKHM1O/fYZ8hj1eUhH GDjdnqMrfmzfXiYNsRtfUc7S+PBJp9RIQ/X7fGpyNjddD0PGmkRZheLtUgZpMUx1QOuv X3UL7Vc+z9qSgHhhKvykd4zKFiBSTKwrR7AyIKJKCMBh66JDfjhIlJHBvvH3L7Jv3dWT F8bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=5nPiJF+WzumJAVVCCb60FHItiJ2GgvRB52oI2WIygYs=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=lOyxHloPrbAZEsLO9mcuwW/9SRoTRWS2SOoo7xdMHG2POubl8oCclgeZAbZLsv2dy2 HuNly0y2tRRSi1DwEHnLpHrur2m+fY1pB0oaZee1bo0TMgYJ3ErcrLV03IsarsZ7rgV1 LP5D2SEnDldsZ4TfT4monbYOizoYDFVqxizMZeSgvpCz2deLtQMUB62+/T5f+dIp6P7h V/yKNXn/W65DHCSgRrbfx8kemcGB2slqJRc3GhPqGoKRL6QaV/OKs3kEpPwHETl69/Ia hjMXVRTHavwL4ubUewuauFA0hnckG14foXTBIQjt2A5DclJ93XSeRG7E/Y3vWLZdr7yg I0NQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2ee941090e9si1239231fa.6.2024.07.07.23.17.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:55 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUx030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:54 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 7/8] isar-bootstrap: Use tar output instead of directory Date: Mon, 8 Jul 2024 09:17:37 +0300 Message-Id: <20240708061738.311938-8-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: B/YuQIuslfbW Do not unpack tar with rootfs content inside bootstrap recipe. This change can allow to remove sudo usage during the bootstrap later. Using tar output requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich --- meta/classes/rootfs.bbclass | 11 +++++---- .../isar-bootstrap/isar-bootstrap.inc | 23 ++++++++++--------- .../isar-mmdebstrap/isar-mmdebstrap.inc | 23 ++++--------------- 3 files changed, 24 insertions(+), 33 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index aa3f6baa..0a1e64e5 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -73,12 +73,15 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -227,7 +230,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 0855875b..4e0ffc0d 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -162,7 +162,6 @@ do_bootstrap() { # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" # update APT mount -o bind,private /dev ${ROOTFSDIR}/dev @@ -196,8 +195,12 @@ do_bootstrap() { mountpoint -q "${ROOTFSDIR}/base-apt" && \ umount "${ROOTFSDIR}/base-apt" + # Compress rootfs for compatibility + lopts="--one-file-system --exclude=var/cache/apt/archives" + tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" $lopts -C "${ROOTFSDIR}" . + # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -213,19 +216,17 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index c41dd3a1..f1ad891b 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -11,7 +11,6 @@ inherit deb-dl-dir FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" -ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" @@ -73,9 +72,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -181,18 +177,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize bootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize bootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -206,7 +193,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } @@ -214,8 +201,8 @@ bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR # we should restore symlinks after using tar if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } -- 2.34.1 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240708061738.311938-8-amikan%40ilbers.de.