From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057983066489225216 X-Received: by 2002:a05:6402:5216:: with SMTP id s22mr7015554edd.359.1643354290487; Thu, 27 Jan 2022 23:18:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:907:1c0c:: with SMTP id nc12ls3612450ejc.5.gmail; Thu, 27 Jan 2022 23:18:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJzjMlJg6WNeoFG2VlyQiP0QvGas93xCIiQyJkigBcGJlsNmm/IkYm2uyDV9XkwaKG8FCQWD X-Received: by 2002:a17:906:5d09:: with SMTP id g9mr5781147ejt.759.1643354289647; Thu, 27 Jan 2022 23:18:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1643354289; cv=pass; d=google.com; s=arc-20160816; b=iimuMMDogWUpur+6OcdskZxFJw04IxwWrGU5DXEU4eC+8jLhaekjEo5OROWIT1Vag4 JtCTN6eNZWK44atQDdOH0+RyhFmXGKwEnyul0ZynYjQlcX6IgqgrlAo72yck8NFGwqiQ wQoxxmmd6A/v6qR5R+zhN9P6MaU+OC7oK+5Pnx6XGSi221m1BQTU5bQpTP0fiz9USbly bWJY5C/QAnckemlvvQtlhg/eyI4sOMXSF6OdMlzR/Lkrm7NuxTXCgjgAJangq1Y3TyN1 TJ9hq68WZdMQkpy55HSSCnS9XmOys6B3rbYIGYzutHJVrFJNFFXCvBAEX3mTeQR0V+Sx FAag== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=ro6NL5zkfy9yy8axQwDJQbJQ3CqqBiOadYGJNtY45Os=; b=WiPW6fyREdzemJpPjRwhv/ULiXM7WC4h9V7Dp5KgCsDisCWOzIVpHFf99O/i0RevxM 1bK6B0L81OsOCVhjympClUa44jWSbfzAHRRtDRwyP1UozdPqhLLlwe1AdTAajeRljZp9 i8l7Y1BdqX/3Ud8G+eKU7p4VvBiKmuks693ok8YQuxuUlvC8rOQrBbB064Sr2U6IfofP Z61ZXNED1AmkRCYVbCwo6lrTZ8ZPM0JKEy+lawc31j+5Y30M+XTjYjTUsjQP5AZ052iR yPORjEpZru6nkJAFH752F4XnQAAnbF1Y2QoVS/wl3ANGjLQGzEWynYGix1opDYA12Hmk 805g== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=QcMOf6Od; arc=pass (i=1); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0e::618 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on0618.outbound.protection.outlook.com. [2a01:111:f400:fe0e::618]) by gmr-mx.google.com with ESMTPS id l16si160406edb.1.2022.01.27.23.18.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 Jan 2022 23:18:09 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0e::618 as permitted sender) client-ip=2a01:111:f400:fe0e::618; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=QcMOf6Od; arc=pass (i=1); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0e::618 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OSGGZcjW22hwA94FGtwXKXXEZb7qWLxFnUxrkFkJvX/nSle+de9O+DizLfHJw+8MdFKAvKBWATegn1Qu9vqb5du8vYeFyt3jVFHCOlJQhj8q3VprtEVjvjr45B1aGlqw3NEZnMeWZWHG6VulHDg4vi+NSIFOrk6xJ+lBlmwykQhyF4lH4zz6A14ViCADMf1IC1sm7u+pPf31kHNcPS1NPX8ZNCqEDRng652qUyUhd+QfN+zinjcwwjDQCucAZ9cLLEik3oCr6Dj5wzBlXTXjUqEEKamFg68A5tseaNV7baINnyKiryN2FwgLx2CO/Sg00t5ruSRmev+mqkk8HXVHvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ro6NL5zkfy9yy8axQwDJQbJQ3CqqBiOadYGJNtY45Os=; b=Atg7A1ZLwRGB173vNOF2Dx/09dRd1u4Ypzq4KKbTVQ68+ilBFkJfkXNazYnW3vxk8l/MzvPlJFIhTQOULZqOsaELK43XyfywgIxKhowafi58rkvZCKwm9rdt64/SaDSNYF8ZcpnVaoXWcTxzQiynSxxO/lJryAxL/q9n94VHUNfau4hPwwW0A20YDBRZwYDVXL1oIRrATA7wtVjUfZF15Pd0LOR1llaMr6WYP416SWvBoZGo9SaheE2fohWThlphmYsE8w8G+Gl4wsl4o0jNbRuk6Ll2S01W2qiXv3NpEksfOc+82FmMpCDYi+XKcr4WEVK8Ej6fP/2UOpcNk/awaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ro6NL5zkfy9yy8axQwDJQbJQ3CqqBiOadYGJNtY45Os=; b=QcMOf6Od2BqRhGwtsG5GXuuqWPrYfkigWDyfsOvSQOaBLxeUBv0VcegmoBeSzQ7vKNpACCrZbDKN8Imfb6k1ZNBALwNhf6KD5rXAPKmG+tDW64HM27FC3rtJyVfNNyeden2aj4IpzVBjs0MWKLIadovyIEuGi46C11+lVh1Lm1nzCZ0MphIYPM8QW8wVbDiGcDyi4UMpiN20n73W/DTM/AxXPJoJ1UeKPI4fvdjWxUINvfJkonakQogJaK1RWRxUFCROsvei65R89FkT1DEf8Box8XXegHQZt5gcjZQPmc1dU/ohDJcWiA0NKFFTfzI9p0hLsnTGM5NOIkFlxbFVeQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:155::20) by AM0PR10MB2865.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:165::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.17; Fri, 28 Jan 2022 07:18:08 +0000 Received: from AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM ([fe80::95cb:ffb4:beca:fac0]) by AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM ([fe80::95cb:ffb4:beca:fac0%3]) with mapi id 15.20.4930.019; Fri, 28 Jan 2022 07:18:08 +0000 Date: Fri, 28 Jan 2022 08:18:01 +0100 From: Henning Schild To: "Schmidt, Adriaan (T CED SES-DE)" Cc: "isar-users@googlegroups.com" , "Moessbauer, Felix (T CED SES-DE)" Subject: Re: [PATCH 1/1] sttate: change rootfs task to cache Message-ID: <20220128081801.3381a43c@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20220127201845.19319-1-henning.schild@siemens.com> <20220127201845.19319-2-henning.schild@siemens.com> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: MN2PR19CA0012.namprd19.prod.outlook.com (2603:10b6:208:178::25) To AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:155::20) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1cad5c55-af19-4f79-db3a-08d9e22e5551 X-MS-TrafficTypeDiagnostic: AM0PR10MB2865:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:262; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CYo0uJ5mN8RGNE54aWsTR9pG/uCdvCwuq3ASbz8eqVqtTDO13dClebGGogjNkAaJiAStSGZc7zKX7KqwSVf30I7YJaOB+CdkDH/zOvYsyVBHHo/n194H6Ch2j+3Hc5TVOEzeNg7/cdKr0vX22keDwY3Lxn9Upy5Gt16v53xX5K5xyIRr/IuldMyFxybp2YrJILitS+nlVlXxes5RhSOBaBM9loH00HJsFMV+0Z9cG6nfW6KIBP+1q4rT59cnvN3kbq4zmTUGY5NMm87TfJDCQBA+CIBXCceC3GE2PPGc7nBZewLAVpTAWOeODFLlWJZKPnw7ojRMiYROxjJ9e9s8FS92CufCfSnfYhIjtpss9yExnrEf0B04DfQEXQ5EQYQeFwq6JxgJEFDZ6kh5pILhbhoqpHSLjJdzFO1Aop/ckG5wf7XX0mDESG9dXyz45w/rw2FwdUs4uPovwL1lVOhVzodpqpb1QPAeyM5d6EmYDBCQ6TNOJm4PTQwIUT2fSQ3cJrSEEDvaWK4jvwx5ZjNDYZqDEkQ1dR1iWWZk1CnczY1SBXWO6Aw7stybjM2Wq1ajYU0Sg2n9G64/jy+AaLtaDKOYKZWiGOqT1Zcem+zF/xOTzq2WtgX1+/9D8FMPC3mVf/52KFPGnHkHy+s7w/fM5w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(186003)(1076003)(6486002)(508600001)(86362001)(44832011)(107886003)(83380400001)(6506007)(2906002)(6666004)(6512007)(9686003)(66946007)(8676002)(8936002)(66476007)(66556008)(5660300002)(6862004)(54906003)(4326008)(6636002)(316002)(82960400001)(38100700002)(20210929001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Tdfa+K6pFabzj7oNQHeNT0rvOCVH4J8IIw7hBl0HZH5GY9cxqz82hYeAf20U?= =?us-ascii?Q?SqczWl92hab59v6NQAgB/KQFKWocimg0Jy/mx3B2UUjRLibJ+4vvPTXVVjoS?= =?us-ascii?Q?RSuM8H6JcxVlFUQbbma/bT0mq9RqfTunPGnq8T0n8nXl2YwxnRVBIrZN8n2J?= =?us-ascii?Q?gefN5b7TEndjyJQwjz1ijGQmwyH/EKFH22wfkd136bgDs+alZMI7kIfFfpR3?= =?us-ascii?Q?6M9b+C7uH1NSxleR0v8xBURpSbt3w5YUEZMdPqaRk6uFHmBoIOjEthlBsJZv?= =?us-ascii?Q?HcU2nPhickACCH6d+Er2o9P9m1K3FwQU5O+VzdgfVRGmNzPhwh33E/LpGG3f?= =?us-ascii?Q?yby+JAeRBDB0w5ekH7IwUrMOPCeGUMCC0MCaxYIMgB1TBocc1LErecCdz8XQ?= =?us-ascii?Q?qfkx4ALVGwusENMOiI+Py+2AHYDbowDkdCqQvsViVl9axjtWrT4MH9yr2skx?= =?us-ascii?Q?7oCdskmMRJ9K+pzx6BjPf4zWtW4Oe7H1HjUYJOqkksPsnnfz9cL2HspqsTKc?= =?us-ascii?Q?ZX/QJEDM8q26YaQyyNc99jY66KEQ/XVdZFQxioSmB6kAO7yP5fofL3ZJHqoR?= =?us-ascii?Q?kW1VJ5n+p0so5/if3bXKbITxAlZKJ8NB0XEy1wMAbzG66Cw6ppO9MUiunr4W?= =?us-ascii?Q?6fLiX9Vhkl0qL9Z0O/uDnXP/MOesFiVTtsjQ7xY+VhcK/jxqkPkkmA2nvNLs?= =?us-ascii?Q?o6JYWjSsf4KDdjRGht/odAkN/bOaEVU6PlzwL+JiegNpzi1FSivZ1Tf29lwA?= =?us-ascii?Q?x0HcJWdN0Ba2ivLWx/Pxosz5f+xSQ752BmHVLVnE4/LrxGgLIHNSMbuC2zej?= =?us-ascii?Q?CDsCRAS6gbFsLbVCzQhLusuu6z6SiZ1QCgCAzXv4Qt6MMAI5ADoqt+bcIco9?= =?us-ascii?Q?Q55G4CwjYKBNPnxcjslqjc4Ke0rg3H1y+0akFRrFx8W33EYZxi+rQfwekvud?= =?us-ascii?Q?rpayq3/U4We8a4iGmYoKQ8+JNiLZm/zB4UfdiYGuebDbIXhfqLJeQIpmJAnz?= =?us-ascii?Q?nkB049/exyaVqwu1B/JldG0KE8vAPtE9/CEHrdohsGKl3AUG2Mj7Sl1HSevg?= =?us-ascii?Q?GMQQAQCB7Fgynn5WmfNSfswge6z4CnjlajTv7sjnek3EGFLVSpXWqZJd2NTN?= =?us-ascii?Q?nh6u4CP/BBQlS4X81pbrHwudz7D95UWt8ThmR08jmoWU8nidkXYcjs0sHHvY?= =?us-ascii?Q?86kvJFhoyzI0vCbe/tBlV6EzD+iEkSuc3JQnewEeNlX0oDv1bnAJsU+7Zgsn?= =?us-ascii?Q?nEmGHvb6S7FoMwdc6kjkrfxKRxvZQKPFnQsw7b4qAy/ujAbeKVTBQAPvzJol?= =?us-ascii?Q?1I3HmA+n2A+zKHlHdyhWLf4T3o4nN6StM5HWImrxxJUxnNAO99sORF6MKOji?= =?us-ascii?Q?xatVsNgNLCI2dr8gRdcGgUuiXjHoHCqDDSvGUg+MKLuqVSPA3G5WPG16ytz4?= =?us-ascii?Q?i+EpG8tfOfnXZdsJNKdHB5qXYZpJFsK6+1I/WtieU1VzD2Hw9Zk3ABn8eYjF?= =?us-ascii?Q?EFit31vFKMOD4YBw+la1m+6aB8P9OJRcSC+XwBKNOQawZr2ud/SuMLnSDh0+?= =?us-ascii?Q?c+V98WuD+boY1gnIgpZGIcKSOkiPjM1WMCoFNB8CINd9FlYQgyDcRtuPeFLB?= =?us-ascii?Q?D3+15FupMtIxOPGTAj2bWWOJ6FkZ9lDSbKLSlSYCoAzT/A+QYVa7ZJ3FyhuL?= =?us-ascii?Q?47ERUoyuLTUgfgFDH+CDI/3s1EP0j7EixyXHWpzHk/CnY+Ny66Z1xzCu27au?= =?us-ascii?Q?rcdujITfpQ=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1cad5c55-af19-4f79-db3a-08d9e22e5551 X-MS-Exchange-CrossTenant-AuthSource: AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jan 2022 07:18:07.9467 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0RHTf/UUdD3Ck1B4UNnHZoJexFllCq//fSw52oFhyth3ps7Rb6h9omxa6NJ6AEY7qh6BEjHwYWQH5eOzn17zAMPTW3lbZoJjN1pF1a9PHaQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR10MB2865 X-TUID: DlCIyS+zdfCo Am Fri, 28 Jan 2022 07:57:31 +0100 schrieb "Schmidt, Adriaan (T CED SES-DE)" : > Schild, Henning (T CED SES-DE), 27. Januar 2022 21:19: > > > Subject: [PATCH 1/1] sttate: change rootfs task to cache > > Typo in commit msg -----^ Thanks! > > This patch moves sstate caching from the task "do_rootfs" to > > "do_rootfs_install". Which is earlier in the task chain. > > > > There are two "bugs" that we fix by doing so, while hopefully not > > giving up on too much caching speed. > > - rootfs_postprocess: > > - image_postprocess_mark: potentially outdated BUILD_ID and > > friends > > - rootfs_generate_manifest: manifest file gone missing in > > DEPLOY_DIR > > > > Since rootfs_postprocess is kind of an open field it might also fix > > issues in ROOTFS_POSTPROCESS_COMMAND functions in layers. Any > > function deriving stuff (like a manifest) or changing stuff (like > > the mark) could potentially benefit from "not caching". > > > > Signed-off-by: Henning Schild > > --- > > meta/classes/rootfs.bbclass | 25 ++++++++++++------------- > > 1 file changed, 12 insertions(+), 13 deletions(-) > > > > diff --git a/meta/classes/rootfs.bbclass > > b/meta/classes/rootfs.bbclass index 1fc487896f4d..e368244ce1ca > > 100644 --- a/meta/classes/rootfs.bbclass > > +++ b/meta/classes/rootfs.bbclass > > @@ -157,6 +157,7 @@ rootfs_install_pkgs_install() { > > do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" > > do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} > > ${ROOTFS_INSTALL_COMMAND}" > > do_rootfs_install[depends] = "isar-bootstrap-${@'target' if > > d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else > > 'host'}:do_build" +do_rootfs_install[depends] += " > > base-apt:do_cache isar-apt:do_cache_config" > > do_rootfs_install[recrdeptask] = "do_deploy_deb" python > > do_rootfs_install() { configure_cmds = > > (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or "").split() > > @@ -270,29 +271,27 @@ python do_rootfs() { > > } > > addtask rootfs before do_build > > > > -do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config" > > You removed that line. > I don't remember all of the details, but there were some cases where > this explicit dependency was needed when restoring from cache to > an empty tmp/ dir. Might need some testing... It got moved up under a preexisting do_rootfs_install[depends]. In fact we might want to move all do_rootfs_install[] assignments there for readability. But hey, i think i will move it back down so we have the actual function []s and the sstate specific []s. Henning > Adriaan > > > - > > -SSTATETASKS += "do_rootfs" > > +SSTATETASKS += "do_rootfs_install" > > ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate" > > -do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" > > -do_rootfs[cleandirs] += "${ROOTFS_SSTATE}" > > -do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}" > > -do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare" > > +do_rootfs_install[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" > > +do_rootfs_install[cleandirs] += "${ROOTFS_SSTATE}" > > +do_rootfs_install[sstate-plaindirs] = "${ROOTFS_SSTATE}" > > +do_rootfs_install[sstate-interceptfuncs] = > > "rootfs_install_sstate_prepare" > > > > # the buildchroot is owned by root, so we need some sudoing to > > pack and unpack > > -rootfs_sstate_prepare() { > > +rootfs_install_sstate_prepare() { > > sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro > > sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar > > --one-file- system rootfs > > sudo umount ${WORKDIR}/mnt/rootfs > > } > > -do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" > > +do_rootfs_install_sstate_prepare[lockfiles] = > > "${REPO_ISAR_DIR}/isar.lock" > > > > -rootfs_sstate_finalize() { > > +rootfs_install_sstate_finalize() { > > sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar > > } > > > > -python do_rootfs_setscene() { > > +python do_rootfs_install_setscene() { > > sstate_setscene(d) > > - bb.build.exec_func('rootfs_sstate_finalize', d) > > + bb.build.exec_func('rootfs_install_sstate_finalize', d) > > } > > -addtask do_rootfs_setscene > > +addtask do_rootfs_install_setscene > > -- > > 2.34.1 >