From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7231094895937585152 X-Received: by 2002:a05:6512:1105:b0:505:73e7:b478 with SMTP id l5-20020a056512110500b0050573e7b478mr1941169lfg.16.1697551549771; Tue, 17 Oct 2023 07:05:49 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:490:b0:500:7f17:b77d with SMTP id v16-20020a056512049000b005007f17b77dls116179lfq.2.-pod-prod-01-eu; Tue, 17 Oct 2023 07:05:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGiOvAQG1UzrbP4DpqG8CM9CDbTTQLNr56QZbIMg4rAxT0EJBZujquRq1Yw7cvaLBfw7cQq X-Received: by 2002:a05:6512:1105:b0:503:8fa:da21 with SMTP id l5-20020a056512110500b0050308fada21mr2124958lfg.43.1697551547770; Tue, 17 Oct 2023 07:05:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697551547; cv=none; d=google.com; s=arc-20160816; b=e7SZnt2OLRilkL1/gRyyo6adIQcGUYUw39qmqzm0RTPqLGFgTu61tudVOGb/j7OMHW 5LjWa/h1Ywld/tTirIdKM2urxqysCkuFe0j8itwFwKREtLyXzMCKoe1BrZxg661Jbncc tghy38NJzax8MWYy9u3J9EzYqisiJUX76C9C4Wivn2lguQekWHQImXaSzhpT384pvC51 yvqFYXq6UWpivRsokrfGyLYOvIFMvGMjFZM1CD0LeGFqw76rY+1M8JlVEdaxXP+Z4A6q a9jYb/waC8xZfGKvDr1YYtL5gzoPjFxudQVlIcVLaKyMG+iyK6oBllWoNiXgfVexL6oH RfRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id; bh=frtXq3JOMYwzKyC/FdlEd7T8yol0mBTBQqsLiBMGftE=; fh=aBcU2rqQYaTz0I1B9FjnMXygC0vCT6ZZq7zN6eet3qQ=; b=rFaro35xXK65hoa8OeIqhsdCfFhifiyZcKj/zLMpCP4XxX6JXnyBNrtdy9u271DDLa CVJonI67gtNYPSNnwv7vsmvtc8CPDMg+rAm+32HZFdiIt/Nf9/wMQ0Ih3TPJAM5dWuAH LNc9eONdYoMBr6Pkcixif+uXAL5NBuwIIrFaz7QYPIDe5Co8qWztsBdxdB9NhEXqAT0c XSD9JPCpgJaVOXvCzbfhjSI5INCivCWiY4wEIh5Yl4Ea9sWSP8dUemsEExGYBZFrRDFL EmUBVW1cQDBjMspZUsgUvE7QW+AlaVdS3KAoDI23s4wyUfpaesoRS92mCmGdTykPh/Y+ NDEA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 11-20020ac25f4b000000b004fbcd4b8b84si62031lfz.0.2023.10.17.07.05.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 17 Oct 2023 07:05:47 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 39HE5iLD021537 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Oct 2023 16:05:45 +0200 Message-ID: Date: Tue, 17 Oct 2023 17:05:42 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 3/6] isar-bootstrap: Use tar output instead of directory Content-Language: en-US, ru-RU To: Jan Kiszka , Uladzimir Bely , isar-users@googlegroups.com References: <20230927043320.6264-1-ubely@ilbers.de> <20230927043320.6264-4-ubely@ilbers.de> <82855302-bd5a-46b8-94ef-77ffbf6b8438@siemens.com> From: Anton Mikanovich In-Reply-To: <82855302-bd5a-46b8-94ef-77ffbf6b8438@siemens.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: tFAmzFDRq6B0 27/09/2023 13:07, 'Jan Kiszka' via isar-users wrote: > On 27.09.23 06:33, Uladzimir Bely wrote: >> From: Anton Mikanovich >> >> Do not unpack tar with rootfs content inside bootstrap recipe. > Motivating this change here would be nice. > >> This also requires moving chroot prepare step to the later unpack step. >> >> Signed-off-by: Anton Mikanovich >> --- >> meta/classes/rootfs.bbclass | 11 +++++---- >> .../isar-bootstrap/isar-bootstrap.inc | 23 ++++--------------- >> 2 files changed, 12 insertions(+), 22 deletions(-) >> >> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass >> index 7dec7b36..09d40563 100644 >> --- a/meta/classes/rootfs.bbclass >> +++ b/meta/classes/rootfs.bbclass >> @@ -73,12 +73,15 @@ rootfs_do_qemu() { >> fi >> } >> >> -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" >> -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" >> +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" >> +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" > zst is really slow on packing but fast on unpacking IIRC. Do we actually > benefit from that (significantly more unpack than pack)? > > Or was it like that before already? Zstd was chosen because it was already used for sstate. Packing counts will always be <= unpacks, so it should be ok. Other feedback will be addressed in v5. >> rootfs_prepare[weight] = "25" >> rootfs_prepare(){ >> - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' >> + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" >> + >> + # setup chroot >> + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" >> } >> >> ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" >> @@ -222,7 +225,7 @@ cache_deb_src() { >> # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and >> # relies on that through out the build. Copy that state information instead of apt-get update >> # which generates a new state from upstream. >> - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" >> + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" >> >> deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} >> debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc >> index d8ac9d00..cb1fc91a 100644 >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc >> @@ -16,7 +16,6 @@ PV = "1.0" >> BOOTSTRAP_FOR_HOST ?= "0" >> >> DEBOOTSTRAP = "mmdebstrap" >> -ROOTFSDIR = "${WORKDIR}/rootfs" >> APTPREFS = "${WORKDIR}/apt-preferences" >> APTSRCS = "${WORKDIR}/apt-sources" >> APTSRCS_INIT = "${WORKDIR}/apt-sources-init" >> @@ -287,9 +286,6 @@ do_bootstrap() { >> fi >> E="${@ isar_export_proxies(d)}" >> >> - sudo rm -rf --one-file-system "${ROOTFSDIR}" >> - mkdir -p "${ROOTFSDIR}" >> - >> if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then >> base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" >> bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" >> @@ -394,18 +390,9 @@ do_bootstrap() { >> "${WORKDIR}/rootfs.tar.zst" \ >> "$bootstrap_list" >> >> - sudo -E -s <<'EOSUDO' >> - set -e >> - >> - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" >> - >> - # setup chroot >> - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" >> - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" >> + # Finalize debootstrap by setting the link in deploy >> + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" >> >> - # Finalize debootstrap by setting the link in deploy >> - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" >> -EOSUDO >> if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then >> deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" >> sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" >> @@ -420,15 +407,15 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" >> >> bootstrap_sstate_prepare() { >> # this runs in SSTATE_BUILDDIR, which will be deleted automatically >> - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst >> + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst >> sudo chown $(id -u):$(id -g) bootstrap.tar.zst >> } >> >> bootstrap_sstate_finalize() { >> # this runs in SSTATE_INSTDIR >> if [ -f bootstrap.tar.zst ]; then >> - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" >> - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ >> + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" >> + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ >> "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" >> fi >> } > Jan >