From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7058179282125717504 X-Received: by 2002:a05:6638:16c7:: with SMTP id g7mr686202jat.261.1643360425837; Fri, 28 Jan 2022 01:00:25 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6602:1490:: with SMTP id a16ls1190271iow.7.gmail; Fri, 28 Jan 2022 01:00:25 -0800 (PST) X-Google-Smtp-Source: ABdhPJwLFgVJlXZQiAEEkTjQn73YuPA2prwF4vwVVpRYzZ9LakFNw4C3p6y2GuqFiK63HtYafoMB X-Received: by 2002:a5d:9d9f:: with SMTP id ay31mr4561336iob.140.1643360425381; Fri, 28 Jan 2022 01:00:25 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1643360425; cv=pass; d=google.com; s=arc-20160816; b=vvAIlGs+GaopjQpf4dmynuZRKkcdpFsSWz8jxK6ztHGGR/phBycJWUGkNQNp/zbH/o zkVq9THDTL7HkQbeCv4O0lIMm1rCzAz3g/w0tQ/k1sUNqK34yY6/BlCz9BmHwJWx7T2A C4i4jxK4YUAFt7bcp2CQEWwPhcVhE6f06aZNWvedtEyZt0kzAyzkfAv2eSbnyjk6o6KV Zn0yMyXw9GpYhSTn5GTtD4LY9u49gYXfdt8kDDwiG2V2cGdIY/AXcizVSwwNYAEPoSaY MowZk3Z3eJu+PuWGpdestLbPhW8RyiQCSswy/qWEAuVJhG8iywDUuJUlTNtqPCLpbxFe wHGQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=j+LKr3XzyiDw+F0RhPLy/T/R0XmFJZ1ZokbSq5m0ZSo=; b=z4zdWcsE3ExJznp0z7AHDosBCvYZRzr2TYL6f98x3bkmtQOjRfHPHZvlhnpyKBeylP LuN/39V6kXWhFojddVdkmWPrR5E/aZbZaaqXwNY6ziZszWESHdznW+6zy1mWyX5p8xiX 8AW3fkmhsk82AnzKzZLh1MXvd+MYve6JWXQ/v//dKXd5INGaRYLJiUItcC1Md/BNVTMV //QXBUDosiwnAN/a/5YvUeJJUvmamQLhB+vLAe52uvxHGYKADZLX1BObXuiEWz8YIG0l 6wGu3ERfp4Aund07SmXDZ6LYpU2jws+mjDrMVNy/KGkV1aVxjcvxL+Yi8Jp9wsQQ/yiC 8FAQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=zMJaZJLi; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.61 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2061.outbound.protection.outlook.com. [40.107.22.61]) by gmr-mx.google.com with ESMTPS id h18si400494iow.2.2022.01.28.01.00.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Jan 2022 01:00:25 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.61 as permitted sender) client-ip=40.107.22.61; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=zMJaZJLi; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.61 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EyHyrJf43rt0xXGEj90K5PC6yKdhMeKsiGhCppP8C5xxCA2v13qgLyfApcakcWPKOZug+Gg92xSjtprR0o7JNg/7Nzn5og9vkuM4Jft7CeKBK2cI2EVE5o3lCKqUGFQrVogQcWZaPVYm3gItiBWf1Nfi9qByIrDmWqEFhRhaoUGIsxHQdk2Pz6TSMYbigKVV3W2ddOXqBJiv2nekhr9ksbzYonftCvX09y+PENQr1021hNF+tbYwf7OMet5GGWMh6rtq6YR/eWYWMWiECl+E5Uq75yjwgn47mUMc2PANmwd278rjI7PIEZ49H1SZ4EoCLwqIzCJ+2lPqSFxxaNNm2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j+LKr3XzyiDw+F0RhPLy/T/R0XmFJZ1ZokbSq5m0ZSo=; b=LzxuXA/XN8Su32RS6klks9yC9zgO9r7y5k0fLoE97ViuXW2UYzMiw2zoGIX5d7FiltE/OPIZYPGPV39tt3k7qFChrl6PbDKMi/L1kxLKU5QSgdbaz4ItOfE0bwd/Ds6KjlKWoKflhm765pRYykwIjyiE22lCAugGmRBUn7QSsz/g/W8zzeoCmBKSKQQhL2hSZ6W0sAGP/M8bozy3wlMykyfhdJRDZaw+OCRfD1FT6TAfQQaFsFCTCSl1x3htCN6wZ65vR4QH0tYLKh2De3azm/t0dORyfHj47j+pdLNSoGHil1FkMJYDx5Nb372qIfqU13tHrHD5CEGZ4m2kAASoeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.71) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j+LKr3XzyiDw+F0RhPLy/T/R0XmFJZ1ZokbSq5m0ZSo=; b=zMJaZJLizNqKF6K/m6FEZVgl4lkif9adO2kZpfZAxuHhf5TfcRBvHYINZxC+ruBWSsPgiO9EWg7aV4Jjcbqpq9NBawyZlL5199J+kQCJFOmRqaAik49qRKbmSrp8pProBmAl/CbOMWWn4wv2dZS9b7hcdCO/nU7L/npkdFUZI2QOwsGJ6zJZ1zdxV56ve9puB+aP8RxubUh6LE3MQX+MyGhhFMuKNxpC2HhgYL+9hhRE4uTp7FZ1WcPkMKCUBaGkTelzPp35hLDBNOGxuDQs4Kjzt5WfVaQJWgXv41V/7ekrhAzZ/spxf7Hhn1jDPoQkR0zS4uk1xyKSjWymw3PyNg== Received: from SV0P279CA0016.NORP279.PROD.OUTLOOK.COM (2603:10a6:f10:11::21) by DB8PR10MB2617.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:a6::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Fri, 28 Jan 2022 09:00:23 +0000 Received: from HE1EUR01FT046.eop-EUR01.prod.protection.outlook.com (2603:10a6:f10:11:cafe::11) by SV0P279CA0016.outlook.office365.com (2603:10a6:f10:11::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15 via Frontend Transport; Fri, 28 Jan 2022 09:00:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.71) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.71 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.71; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.71) by HE1EUR01FT046.mail.protection.outlook.com (10.152.0.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4930.15 via Frontend Transport; Fri, 28 Jan 2022 09:00:22 +0000 Received: from DEMCHDC89XA.ad011.siemens.net (139.25.226.103) by DEMCHDC9SKA.ad011.siemens.net (194.138.21.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Fri, 28 Jan 2022 10:00:22 +0100 Received: from localhost.localdomain (158.92.8.115) by DEMCHDC89XA.ad011.siemens.net (139.25.226.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Fri, 28 Jan 2022 10:00:21 +0100 From: Henning Schild To: CC: Felix Moessbauer , Adriaan Schmidt , Henning Schild Subject: [PATCHv2 1/2] sstate: change rootfs task to cache Date: Fri, 28 Jan 2022 10:00:09 +0100 Message-ID: <20220128090010.13589-2-henning.schild@siemens.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220128090010.13589-1-henning.schild@siemens.com> References: <20220128090010.13589-1-henning.schild@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain Return-Path: henning.schild@siemens.com X-Originating-IP: [158.92.8.115] X-ClientProxiedBy: DEMCHDC89YA.ad011.siemens.net (139.25.226.104) To DEMCHDC89XA.ad011.siemens.net (139.25.226.103) X-TM-AS-Product-Ver: SMEX-14.0.0.3080-8.6.1018-26680.006 X-TM-AS-Result: No-10--13.624900-8.000000 X-TMASE-MatchedRID: PbZzGcrqIHDD1iGBW1nkVIdOR4lcXha0L9pNzNuyd9YdyHiBnyiVv0+c rEA4+nhZzP9LEqj2YngxE7q6mglq7rWbe6WXLfSr+PaEqEu8Ud/oII8zZ17GT212v13Qb606Agv M6h73BtqDy8d72OLzYoS96wkOlNPk7DYHnv0JPMDdghnhCkMrEE85Ae+vNy5zPwKTD1v8YV5MkO X0UoduuY5JUK9UdYkn0fNMT0mtrLmeIInjW/AwirvmMHkFu/+tc84giXPqvEZwsULORIYX430tC KdnhB589yM15V5aWpj6C0ePs7A07Xi4XEoPXecxDvRAQ3LRQGC3KHxZhG5ZbNGw933IoNgTcxN7 is85EwE= X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--13.624900-8.000000 X-TMASE-Version: SMEX-14.0.0.3080-8.6.1018-26680.006 X-TM-SNTS-SMTP: 10391F95B91C0F0FBFFADE0ADC91E92875D198C2B7226C2C08ABB632069349A82000:8 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 149c3cab-e559-4aa8-cf7d-08d9e23c9e3a X-MS-TrafficTypeDiagnostic: DB8PR10MB2617:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yyBNWuNzsgKijHatFx4nm/qhTnTEsWfWqM312OiesOUI2akBNvAAGLBSua8lKipQtYev5HnfUTdP8D/DMQpb7LWuK0vyBLWdOvM67DirbhR8Xckq8MJUdctfkmr4PtOlWHq/uuD3zGcad/9yBJyFkZskaZgaYRXSCTmxWEXcL+pqL221hTu03+6Rv1s9ZCRiSfI/4HL/TNPUWV6+mNP8haTnW5XLzMCVT4w9gmZYppcm+S3Sl8bPVNtWox7PMGV/+ReNavmTMzj/iNcXkdG3z3U+BfILDZtnAiidRGrABL8LHcVlzcm08iyzhBTujdjSpqDNQgsG5q3APHzMcVikihyTlkfrW5JH1JNWyGxONnZbRzlzifUF76B5lXuBwGCLGSkEB+FCX+H33cmvM/60N65vklqFYZOK0Xd2NOhrKjpnHyClhfw1L/QSplgYfclqaYFK2NAvc9hMtkw3p+lMMSTVGhH3DBzv95ntbLeZc4P8ExWtihq4CM4tgl4qA3ZXwofMy5sTcg8/B0tDsfzhrkzJnlonFG0HdHOATKHfLdTe4tTmk/gxyRo3fjyuvKnSrIMZ5ttTnnSQPWsbGBEBMU6uEtc2d+cqUgzHnWYhb5bb8dd/Stf2rKMRboxyLh4iZWrt3VrMJ+HswMVghmFhIJ0vzOp4zXRK1zSGY9bDjILWaDpjxMhVHTdbhlfrLpIzejg9c/OVczayK9sq27yOXQ== X-Forefront-Antispam-Report: CIP:194.138.21.71;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(2906002)(36756003)(26005)(186003)(1076003)(16526019)(47076005)(83380400001)(54906003)(316002)(356005)(6916009)(81166007)(82960400001)(5660300002)(70586007)(44832011)(6666004)(508600001)(82310400004)(40460700003)(8936002)(4326008)(8676002)(956004)(70206006)(107886003)(2616005)(336012)(86362001)(36860700001)(36900700001)(20210929001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jan 2022 09:00:22.9346 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 149c3cab-e559-4aa8-cf7d-08d9e23c9e3a X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.71];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: HE1EUR01FT046.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR10MB2617 X-TUID: /pc5VzyX64YZ This patch moves sstate caching from the task "do_rootfs" to "do_rootfs_install". Which is earlier in the task chain. There are two "bugs" that we fix by doing so, while hopefully not giving up on too much caching speed. - rootfs_postprocess: - image_postprocess_mark: potentially outdated BUILD_ID and friends - rootfs_generate_manifest: manifest file gone missing in DEPLOY_DIR Since rootfs_postprocess is kind of an open field it might also fix issues in ROOTFS_POSTPROCESS_COMMAND functions in layers. Any function deriving stuff (like a manifest) or changing stuff (like the mark) could potentially benefit from "not caching". Signed-off-by: Henning Schild --- meta/classes/rootfs.bbclass | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 1fc487896f4d..78fb2cc71350 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -270,29 +270,28 @@ python do_rootfs() { } addtask rootfs before do_build -do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config" - -SSTATETASKS += "do_rootfs" +SSTATETASKS += "do_rootfs_install" ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate" -do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" -do_rootfs[cleandirs] += "${ROOTFS_SSTATE}" -do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}" -do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare" +do_rootfs_install[depends] += " base-apt:do_cache isar-apt:do_cache_config" +do_rootfs_install[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" +do_rootfs_install[cleandirs] += "${ROOTFS_SSTATE}" +do_rootfs_install[sstate-plaindirs] = "${ROOTFS_SSTATE}" +do_rootfs_install[sstate-interceptfuncs] = "rootfs_install_sstate_prepare" # the buildchroot is owned by root, so we need some sudoing to pack and unpack -rootfs_sstate_prepare() { +rootfs_install_sstate_prepare() { sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar --one-file-system rootfs sudo umount ${WORKDIR}/mnt/rootfs } -do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" +do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" -rootfs_sstate_finalize() { +rootfs_install_sstate_finalize() { sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar } -python do_rootfs_setscene() { +python do_rootfs_install_setscene() { sstate_setscene(d) - bb.build.exec_func('rootfs_sstate_finalize', d) + bb.build.exec_func('rootfs_install_sstate_finalize', d) } -addtask do_rootfs_setscene +addtask do_rootfs_install_setscene -- 2.34.1