From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057983066489225216 X-Received: by 2002:ab0:986:: with SMTP id x6mr2645935uag.50.1643314740848; Thu, 27 Jan 2022 12:19:00 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a67:e085:: with SMTP id f5ls1340266vsl.9.gmail; Thu, 27 Jan 2022 12:19:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJy0lFdTBupM99TP45SxKyTx3t/4e4K43S8SGg6MdTPRmkIbYtLmsCQmMqF0EIgxZ7W+PBFy X-Received: by 2002:a67:22c3:: with SMTP id i186mr2347026vsi.44.1643314740143; Thu, 27 Jan 2022 12:19:00 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1643314740; cv=pass; d=google.com; s=arc-20160816; b=bKcLufyzwN0oaC4CjDRE/vOxzijrOIpvnOPhtXx1VoVi1DVspu0xlZEej7NAdw9p82 PCHVdODzjYErBlPZmlcOnzbiQ5Vp/dqSfvmry9WqcQwKe+gkoFAVBvnaC6iggiNyb2wc dh3JX/1FD6BOFLfC/8uQpykjrDRmErBJwgC/cEKbqGG0fD0ABrCX9M9/wOwblmC1Ohpa hTu+aEwqjWgq3LhbUJ0HkQo4z/yXS+czNynac7Tlf+b5eOfK1SjCrbEY76LjvS5hBVGm kFoCfcuwSxgxSYDHhIp4O4ntnn+wJg9BnjaTgBWaajUm/ZXZ57U+slzpg47MK/FhLxy+ v4lQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=SDvWc1yPrr5zyLCmYLej/XBKrI1TOaI29egBHnrlrTE=; b=gqhqLxyD1hmC1HmTEhrO/1S0gw/LvTymhSWM5xlZX1HpCTHR+a/nNqUbcLL6IOEV+1 kFLtMkAyrghREm+XYQtus/9JFM7z2HbBeqktFmdqXFhMRsgKCXELl4salNOz6boWnoN/ kqCAUNkcJOe/Tz6nsaW1XQQ5GjPiI62QK4deuYKrZoNxxIv8Y/MOQRbmLcwbXMQ8d4Cr Uwry8IdS3BNS3VSIcBQgvsNb2HybQeJS4SCzNNSVuXOOQMTPpHDOOxHR6jLyJmiR7wn/ xM6umybVSXcqT1xh0d50Fn1lEMBiZKVVaPS8L5Jij7HEaNKduCSBdbwDOiFecWSSWcSW /RMA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b="McZE/+AZ"; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.8.77 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80077.outbound.protection.outlook.com. [40.107.8.77]) by gmr-mx.google.com with ESMTPS id w188si729627vsb.0.2022.01.27.12.18.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 Jan 2022 12:19:00 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.8.77 as permitted sender) client-ip=40.107.8.77; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b="McZE/+AZ"; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.8.77 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fMOV0XDmLjiJcwDSXUWz3ZUZwpnq5iRj7BWFjfFNgfRvcThOkxApZID5VQ3sTA1FSfn5cByMzZYBu+uMdX+8yrGGEWQV57F63m3aTczmbEcstTWqOi/NkMu1Pz3dxxmKSKz3TM3AMnB6HjBdjGFIvh2aVndkEfL0sWB266mL7hTdj+2qf/fmQbpKtPc9TE4d7Pwt6ly2YslbR+0K7Rrow2FjQr9U6jfBVIQyZARmXuz4Qnli2HkRFNqI8GkFe8E/tojgQ2Ne4RtybNQIdXqIGBGLukXFDxyMOC/EBTYKqH+g8f6aW1HGN+kFJJ8SvHG74NK1831qTwQTfIVeYtEg/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SDvWc1yPrr5zyLCmYLej/XBKrI1TOaI29egBHnrlrTE=; b=cMHomRY6VGlgXDwtJ5C+xlajxiLW7bNxfyshI/qZfIUM5+QkiXEarC2cqo+9yazWhsrhnvfjw2qxYxjq/30Y1k4Se06EavQSgna/JIluKGfI0cLlp+kO7zPjITqnQjlsrCkrjTdlqt52sZk3KrGvJNEhpz4ghGGq+6LYXSuWHdn29bFR0P3RXO1xrWqlVUolC1y9Fzt/qIbLG/5dJVx9HX1fNeYmJN0RZKecddU7riOTcMlBSVn4XHmC4oOf4Mn/jm57YLX6spJTIV+IX5jagMMdGiXLyaXZ9TLB8RrlzYEXhOCyKABTeUJfEYlp8fwnoJnZ9LduFYg8/cl6XQnTlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.70) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SDvWc1yPrr5zyLCmYLej/XBKrI1TOaI29egBHnrlrTE=; b=McZE/+AZnRoyS9vzOR40boA+9cHmL0+a+1RJ7yiVTPr31RZlC7pxEV048QLvIVE7vhTBwE8iRNGedk/mA2nhIIKAOxHUnFE84aAtEw4C0ikfAG8+yzi2/yGQyUHajzGBWcKSqKhrJ+gcxftYRlqOYzn8H7ZSWofrOhYoioqlvkjK4T3QbNrMjmVZ+qtB0Qhha3Ro7Q3MY/dARHDFo/9wMqDqDNM+4LNw85ST2+n0b3t180JmV4CIKJEE1hj4HzuaZ3pIoFBkxBo6witLID04WR6HRkVmPaJP/bFEdqNVFouRG15if4QteDzUWDLde1NdyM/IpXjHIifAG27U/VnKuA== Received: from DU2PR04CA0228.eurprd04.prod.outlook.com (2603:10a6:10:2b1::23) by AM0PR10MB3714.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:154::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Thu, 27 Jan 2022 20:18:57 +0000 Received: from DB5EUR01FT021.eop-EUR01.prod.protection.outlook.com (2603:10a6:10:2b1:cafe::60) by DU2PR04CA0228.outlook.office365.com (2603:10a6:10:2b1::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.17 via Frontend Transport; Thu, 27 Jan 2022 20:18:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.70) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.70 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.70; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.70) by DB5EUR01FT021.mail.protection.outlook.com (10.152.4.245) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Thu, 27 Jan 2022 20:18:57 +0000 Received: from DEMCHDC8A1A.ad011.siemens.net (139.25.226.107) by DEMCHDC9SJA.ad011.siemens.net (194.138.21.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Thu, 27 Jan 2022 21:18:57 +0100 Received: from localhost.localdomain (158.92.8.115) by DEMCHDC8A1A.ad011.siemens.net (139.25.226.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Thu, 27 Jan 2022 21:18:56 +0100 From: Henning Schild To: CC: Felix Moessbauer , Adriaan Schmidt , Henning Schild Subject: [PATCH 1/1] sttate: change rootfs task to cache Date: Thu, 27 Jan 2022 21:18:45 +0100 Message-ID: <20220127201845.19319-2-henning.schild@siemens.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220127201845.19319-1-henning.schild@siemens.com> References: <20220127201845.19319-1-henning.schild@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain Return-Path: henning.schild@siemens.com X-Originating-IP: [158.92.8.115] X-ClientProxiedBy: DEMCHDC8A1A.ad011.siemens.net (139.25.226.107) To DEMCHDC8A1A.ad011.siemens.net (139.25.226.107) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a647d8c6-be15-4017-f5f4-08d9e1d23fa5 X-MS-TrafficTypeDiagnostic: AM0PR10MB3714:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: be7Oe23CK755emjEBAhxhx8+6smbGI0hdiXtVOwKBHB8ja1+40YazkMvMkwIRiOp4tfSvrRftSo9ZzByb/I7pj3yJ82sr5Nnw3vgTdakMU+wLqxhRBmg6vZpZvTd1hvWkiHsHMCisXxAPLK2RUrDjDpCY0vkLYCkCvLb+iRknIpcHkTfjBm9K1b9FD0iQpFk8dltKoequWoALGf7XY3dbtmtW8OaNgPbMBd6jw5yOTq7N7cVb60t5ouMgeDgpGzyh1G1EJYbq4mf8F8WvKpC+P3e8BEeH5h960sDJ/N348+nxOY96lp9xhVyjUOS2ctqJpGOrVAnjeOx2abj2X4G3KllumR6IRwGR6H0ijRuZV+w00prRdMyqoHKyMWD7VkoXdlVKp4kpFUElD6mzw/+BvUPyYjcFcDN5yYieeX/ggETZQv1Icq3l+0MPhP9TPQgg5nI6m7E2BF49ZOr7+IijhKuyrHSled3x5kjIzpPKGEaFYSjI1wKokJy7/A7uluP7Do2ltCsnpAtZM446fgJbVfbdS1N4kD1aFqHfR9D7JdsYQT7uWhvws94nbmMcMnmXh41Yb/ETjZFwBv8foKbsw6l9RnKmGfPhGQcz09P6V7W8FsS+Xe42h/aPdgFsE7cA44O2Mjt8zK2l4OD+lzT2cb3nV4ed2Nckv1GsC7NV2gDr8jz8KeJ7fwVZopni5SS X-Forefront-Antispam-Report: CIP:194.138.21.70;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:hybrid.siemens.com;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(47076005)(83380400001)(82960400001)(26005)(2616005)(186003)(956004)(1076003)(336012)(16526019)(36860700001)(356005)(7636003)(7596003)(6666004)(2906002)(70206006)(5660300002)(8936002)(70586007)(82310400004)(40460700003)(8676002)(44832011)(36756003)(4326008)(54906003)(86362001)(316002)(508600001)(6916009)(107886003)(20210929001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jan 2022 20:18:57.6233 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a647d8c6-be15-4017-f5f4-08d9e1d23fa5 X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.70];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR01FT021.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR10MB3714 X-TUID: UMNlG3nvi21u This patch moves sstate caching from the task "do_rootfs" to "do_rootfs_install". Which is earlier in the task chain. There are two "bugs" that we fix by doing so, while hopefully not giving up on too much caching speed. - rootfs_postprocess: - image_postprocess_mark: potentially outdated BUILD_ID and friends - rootfs_generate_manifest: manifest file gone missing in DEPLOY_DIR Since rootfs_postprocess is kind of an open field it might also fix issues in ROOTFS_POSTPROCESS_COMMAND functions in layers. Any function deriving stuff (like a manifest) or changing stuff (like the mark) could potentially benefit from "not caching". Signed-off-by: Henning Schild --- meta/classes/rootfs.bbclass | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 1fc487896f4d..e368244ce1ca 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -157,6 +157,7 @@ rootfs_install_pkgs_install() { do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}" do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" +do_rootfs_install[depends] += " base-apt:do_cache isar-apt:do_cache_config" do_rootfs_install[recrdeptask] = "do_deploy_deb" python do_rootfs_install() { configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or "").split() @@ -270,29 +271,27 @@ python do_rootfs() { } addtask rootfs before do_build -do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config" - -SSTATETASKS += "do_rootfs" +SSTATETASKS += "do_rootfs_install" ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate" -do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" -do_rootfs[cleandirs] += "${ROOTFS_SSTATE}" -do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}" -do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare" +do_rootfs_install[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" +do_rootfs_install[cleandirs] += "${ROOTFS_SSTATE}" +do_rootfs_install[sstate-plaindirs] = "${ROOTFS_SSTATE}" +do_rootfs_install[sstate-interceptfuncs] = "rootfs_install_sstate_prepare" # the buildchroot is owned by root, so we need some sudoing to pack and unpack -rootfs_sstate_prepare() { +rootfs_install_sstate_prepare() { sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar --one-file-system rootfs sudo umount ${WORKDIR}/mnt/rootfs } -do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" +do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" -rootfs_sstate_finalize() { +rootfs_install_sstate_finalize() { sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar } -python do_rootfs_setscene() { +python do_rootfs_install_setscene() { sstate_setscene(d) - bb.build.exec_func('rootfs_sstate_finalize', d) + bb.build.exec_func('rootfs_install_sstate_finalize', d) } -addtask do_rootfs_setscene +addtask do_rootfs_install_setscene -- 2.34.1