From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7182796543154454528 X-Received: by 2002:a05:6870:6c15:b0:15e:d9fe:f5ec with SMTP id na21-20020a0568706c1500b0015ed9fef5ecmr1440030oab.46.1674199870368; Thu, 19 Jan 2023 23:31:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6830:2714:b0:686:4846:e9e4 with SMTP id j20-20020a056830271400b006864846e9e4ls741235otu.0.-pod-prod-gmail; Thu, 19 Jan 2023 23:31:09 -0800 (PST) X-Google-Smtp-Source: AMrXdXuh3qZlak7m4rkc1ISpkii7h4XuMSno5Bxts7/2zQnA4PjU09fu5rpRXcYveCDWs1ZmrDAb X-Received: by 2002:a9d:63c5:0:b0:684:9f9c:38c0 with SMTP id e5-20020a9d63c5000000b006849f9c38c0mr6455904otl.12.1674199869830; Thu, 19 Jan 2023 23:31:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674199869; cv=none; d=google.com; s=arc-20160816; b=JP5aGpC6q0RaCsoCLCIPyTo9qMCIf5Pt7LezvYEEuiwK93KnXHm2ynMV9dqSfD3/03 E88id3aUi1liW1ZQ3xUruriW7kXO5kMGQtfamFrwW6QDXObWLYgJM+n7S0AS9LPgYXvl 0Lwh/Gj6VxD/Bsd0SEhEP/WELR0TlmYrTAccyMcMfsCbnGEl2fYkLOZlScwXW6Avvzui lSkG8uIi9b8Mpc8jEnnGHYjMm9T2ucLbyw+QApgC7kIHZEDrkEYrLHMk05ZFTYbqBtai kJsnmjzdS1OecBXHHS6EaL+knm7ruzX+47X8IKPKU5DdiisN7g5COi8OkW+qdU0ULs/a r8cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=BnmhTjC1j0c0UG5YhTRoFpNO13R4UfVmMNzpBOhkQHY=; b=N5bkT2pFHm29bPq+goru52NAytLH+7qvyvnYuDPGWLIoXaHT2ylQcUNHQ6oCQrdvP5 xuTHeXAqLz7Uq0IW78kjbiSqE0W5U9DrIwJV9LjkLLJqMn8r0pg28rwkpAg8Z+zzBIRy m0p1mQl91Zq6q3HYJ9PDEHFRYGBkNco4xft8oF25aW/9brlQQvdpoi7J8gbrCYRADgZ2 DC2B0SwGPGhc1j1IPR3uh8CwIM/+PRshQvG632/on5glK+U5fYoyI0uVrLktQ+ucVDK2 yuWnhYkgGksxproBIZxvBneAjkb4ISsdR1xwgEx5gydj5aQpya+KRte9IqwD2hDb21j8 VkIw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id u13-20020a056871008d00b001480308ea6csi4857661oaa.0.2023.01.19.23.31.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jan 2023 23:31:09 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 30K7V6DU001626 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 20 Jan 2023 08:31:08 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v3 5/5] Lightweight copy of rootfs directories if possible Date: Fri, 20 Jan 2023 08:31:06 +0100 Message-Id: <20230120073106.5453-6-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230120073106.5453-1-ubely@ilbers.de> References: <20230120073106.5453-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: DEKWJJhtnUb5 Some filesystem (such as BTFRS or XFS) support copy-on-write feature. In this case, data blocks are not really copied until modified. Using `--reflink=auto` enables this feature for `cp` and allows to get a 2x..3x speed when copying files. Some measurements on HDD and XFS (~900MiB rootfs from isar build): ``` real 0m31,828s user 0m0,369s sys 0m3,403s real 0m13,611s user 0m0,156s sys 0m1,267s ``` The solution is taken from Roberto A. Foglietta patches and released in a separate patch here. Signed-off-by: Uladzimir Bely --- meta/classes/imagetypes_container.bbclass | 2 +- meta/classes/rootfs.bbclass | 4 ++-- meta/classes/sdk.bbclass | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/meta/classes/imagetypes_container.bbclass b/meta/classes/imagetypes_container.bbclass index 436a0051..50645aea 100644 --- a/meta/classes/imagetypes_container.bbclass +++ b/meta/classes/imagetypes_container.bbclass @@ -40,7 +40,7 @@ do_containerize() { "${oci_img_dir}_unpacked" # add root filesystem as the flesh of the skeleton - sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" + sudo cp --reflink=auto -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" # clean-up temporary files sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 517185e8..19ce998f 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -78,7 +78,7 @@ BOOTSTRAP_SRC_${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTF rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -218,7 +218,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass index 79e01a19..3feada21 100644 --- a/meta/classes/sdk.bbclass +++ b/meta/classes/sdk.bbclass @@ -82,7 +82,7 @@ do_rootfs_install[vardeps] += "${SDKROOTFSVARDEPS}" ROOTFS_CONFIGURE_COMMAND_append_class-sdk = " ${@'rootfs_configure_isar_apt_dir' if d.getVar('SDK_INCLUDE_ISAR_APT') == '1' else ''}" rootfs_configure_isar_apt_dir() { # Copy isar-apt instead of mounting: - sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt + sudo cp -Trpfx --reflink=auto ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt } ROOTFS_POSTPROCESS_COMMAND_prepend_class-sdk = "sdkchroot_configscript " -- 2.20.1