From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7186996775974076416 X-Received: by 2002:a7b:c30e:0:b0:3da:f751:5ecc with SMTP id k14-20020a7bc30e000000b003daf7515eccmr1735263wmj.11.1674462426605; Mon, 23 Jan 2023 00:27:06 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:d211:0:b0:225:6559:3374 with SMTP id j17-20020adfd211000000b0022565593374ls9672846wrh.2.-pod-prod-gmail; Mon, 23 Jan 2023 00:27:05 -0800 (PST) X-Google-Smtp-Source: AMrXdXvew+3z5Fe43sEOAkCSz4NfEL4IxIq6e8+MKstLi4q5pFbtMw23lgmt18RhqKiZSzNyA0gA X-Received: by 2002:a5d:4384:0:b0:2bd:f8c8:9786 with SMTP id i4-20020a5d4384000000b002bdf8c89786mr19877591wrq.64.1674462425571; Mon, 23 Jan 2023 00:27:05 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1674462425; cv=pass; d=google.com; s=arc-20160816; b=EYfcgK6UL8jidWh9DtZoC+stBPXq0dsLLczCx1hNRxF6pSe9F5/S4s2AR5Tg1M5e+Y 7v+D8gmvBLiYG3pu59eAXbY01zw19GXbWWSiLUfa0dc3ku8+xt+IMIpl0Y6fwQSUHEbu 0/S1kewl99pwQfvDgywr7DLpUjYMrAV5/BWKEIzy993CcudzS35XFQktgQvBCRBiPGF+ i8HTQH39qpIN/U/Zs0W6btDCfLLhqizd7LtDSOmsTaX9NeTaCc0ODEQGDCgIukdZyem6 U2K+Y/ja/fgoPxVr6lpeAcXObSN20lndzPGMPSBPMJV/t9barYbgpxH7aa/8MuPA3jEY cqSA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=xvVMqSu6mVUJIcX4zwbmUBBVTrrOw03cVFKi0HCAWtY=; b=isJvWrFTjfCtoZVXXUCuKEUgBW2gTW3lNFgQbM0IipnGlfYA1/3rYyWnJOhpzAC+ql GDTchc6NM2936IVeAEg17xxcclbcHAfe2CxcsR3roGHdIw2Wy0NZq+0yCiG+H4rA1S4G I94RJfp07R6uOh1TkJxBS0vS6Y8DzYZsIaXyLZep3wwKMB4D2YsnERrfxf3+eAIqTSyG DrFk0otbbBfuikWzTMuA7cvoy6Y+MIYokVFrchWxWmqf4dt/xlVCnYQD1reur/Y2LMQO JEqtrJp3su/VkLNca6e5hNYc72OUYlXfTqMaxQdH9TCwA/DjS9tXROfa+s9YFYo288rE rBUQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=UO7XLpp1; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.20.79 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2079.outbound.protection.outlook.com. [40.107.20.79]) by gmr-mx.google.com with ESMTPS id d10-20020a5d644a000000b002bddc018216si1284043wrw.1.2023.01.23.00.27.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Jan 2023 00:27:05 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.20.79 as permitted sender) client-ip=40.107.20.79; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=UO7XLpp1; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.20.79 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J3lldSLIo+1dG6SAZsPh7zB4TD1uXR+kQvHdwr/5qM4M+HPsK8MjRafU1h70qPtmYjYnlapTKQNz5Q87bfFYBK5td0pdlfR3AZH/tfsJngmVu9myyJ46qgMicCRkEglzpYDGcU2pvxAnm4mbsp+nVTejoj/wYbNvtH4sPrSnXWBOq6FqjyEyQpu+7PP+GsvMj24lctLvv+DW5QbAuUyEOf/YJfyHKYokqt52IUwK9sGkhS0YTWMKJckWp1UGxXWXfcOZlXrSZ43cZd329h75QHqlRaDYOO42QJ0XBZUMHS8trwclYL45xGZLE+AQNLji4N71pt+M6NhrSyjZIXQT5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xvVMqSu6mVUJIcX4zwbmUBBVTrrOw03cVFKi0HCAWtY=; b=G76D+br7cfqnzbhG5VtMB8KQfFB4i1r/eTJMqWMpimNZPZcig1++hqHO/dJT5Fcom5G11x1GhLkvUgsKFnXx+IHHdytaH77oG1YWTtUfvfRLhumZD0YWBH50Yzt9y/DVk290SyYRaPLFMstV5JYkUUwNNHy2tgdqBCzkTd2G7l5MXLfKGtnyA4PstX2GJB+60zpc4YkXYBzrE7szcxvf873k1wnxK6ipHZyHs7hueAt6g/yNzMQkHDX0O5AIDsq5ZYAVBTL1oR9/Khd+Vov9TCgtleY75ZwG3BAPjJhCDUPXqtQy2kbex9hNtAtH+5dwlM7Ykgnp9P0RVwedCpzJgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xvVMqSu6mVUJIcX4zwbmUBBVTrrOw03cVFKi0HCAWtY=; b=UO7XLpp1AbBfXs72orb0Q2GRWkADqtItR7G0159UNW/qSB1kLXpPUQ0S2SIwi8BTj6d3Cjicy8mZjUabBBXPJEINPSh+DIT5idz7c3E+e1z/JUBM1GXEKmuAIiksT5wS+G9g43qvefgddx2qDHwtDmVY4mRqxYtoj12aOJhGTXayT2FxOwSdQTghfX8ZrtD65hbBlMGyzL6RnFuCttnvUNv/fVDgIM1yGDcaeLmZdYeQkM1WPaOPU4gMSxXSHQ9XAAMIRf7Ss7LuVS7bBAHZqjEIROe5kiJDG4u+LOt/tDXJsvLKE2JLZMTnQXvWd0lbv/me44HwBnmG3oJ0GA6J/w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by DB8PR10MB3642.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:13a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.28; Mon, 23 Jan 2023 08:27:04 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79%4]) with mapi id 15.20.6002.033; Mon, 23 Jan 2023 08:27:04 +0000 Date: Mon, 23 Jan 2023 09:26:58 +0100 From: Henning Schild To: Baurzhan Ismagulov Cc: isar-users@googlegroups.com, "Moessbauer, Felix" Subject: Re: [PATCH 0/8] CI rework of gitlab fast job Message-ID: <20230123092658.2275d928@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20230110121748.14917-1-henning.schild@siemens.com> <20230112161250.78000195@md1za8fc.ad001.siemens.net> <20230113004315.65f968dc@md1za8fc.ad001.siemens.net> <20230113181401.3b8e5807@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR3P281CA0031.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::9) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|DB8PR10MB3642:EE_ X-MS-Office365-Filtering-Correlation-Id: 8cbc19d2-34d5-4326-214b-08dafd1b9b5e X-LD-Processed: 38ae3bcd-9579-4fd4-adda-b42e1495d55a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KtGKELI17RU+tIQL//6TnmSrQuEl02KjIQAiK1hERrQPZa5wkUYEaz46jWEaBDi9s1HBPFviyofU0JSJMvMkaVYdmg1irlGo2qicOCYsXsygZyrPtx3XVljKwdiZFm7X+uy2jKOyV3v+/CbS3NVqPWw4Mc0QrwdJ4B0noPzyNBugySla4MH7dwqeMiprQSjdkXF4XfhDS/k3myEy/27Rx/7BOWxtQnm8p81vljlyDnEVjj91I4KtEr2LcEUJF7MkOFl8RGTAwjwEdEdqn1RWwjra+GtzFWF1yX5+943vxqVXa7Lg4myd9YJ8NhwZtuGaXnvehfRmiaCmr6YNY0TWXmQROVvUkM8AnrL1nOd3IlYwuBQXWrOMClna8vyxvxJ9dOZWOhofGASB59IclRtNBJQDMqTUq8PTPhU2WJMQ3DW3UPROpbo7I7lmIyFrEHgiA7d60YAeQOwxv55s0VV3pfRGSn2TfXkz9YjrvPYX2luy2bJjdMcTLUCJCAzJvisf7otgUHfgQ8DjiIvjSMVRJwX5DuvV5Wa+M38iw56/CGyQCZYGaekj2snfZZq1VtNXxtbU01kp2GR6+Dpn/iW9GyowBswGRdFi5Ik+3St4Vr0HtczJvxDkNdwBRtbewsUFqoSksFd7XytOMhQ7HWDrsg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(136003)(376002)(346002)(366004)(39860400002)(451199015)(8676002)(66556008)(4326008)(6916009)(41300700001)(478600001)(1076003)(9686003)(66946007)(186003)(66476007)(86362001)(6512007)(44832011)(2906002)(316002)(82960400001)(6486002)(6666004)(107886003)(5660300002)(6506007)(8936002)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Al7W6HZ8oTxV6UnbP4LbwecsDXEZ4EBqLgWqJD9kThBzkmY2wNa7vmnv/ilq?= =?us-ascii?Q?R+QinNHOwjzLx0YQLxoUNq372WFv+1liLGffbtZohzu6LbD8og9jj90KKQ9w?= =?us-ascii?Q?EadD7a8C/uWlwJNTyBwjoB/YnxSv/Jo6T+oy79MUXrqjIdxAcSle+ABnLsZn?= =?us-ascii?Q?WDty+iyfSZW2U8zGoU9UAq1bzGo3S0Scb5cmoidujOabrdsaWAugBS1kcgTQ?= =?us-ascii?Q?ez1k5k5VZr6Wu/Mi4U6Ue729q9982SbfW/jCcIv9H8kBRF4M77pkIWjM4vqv?= =?us-ascii?Q?l5HhTEIfGesluHdAoDv47EQLaQ0FvOLTO0tTBwoKq64zcgSxUtTPEwK3rRAW?= =?us-ascii?Q?SYBiYRk2NO4nvhaBFppUaPBbn3eWKEsJMsVFoS2GOdH1MtQ+A9N7MhWJz7WM?= =?us-ascii?Q?sVoaOT+1itDQuSx5JpDM8YMNESE12zTbidkgU1bswaklRX7n2xzkZgm/RPHJ?= =?us-ascii?Q?LG3SxADd5Z83wrcdDxvmCvoRZm1UuTvSVOijm8GuoDibdBvJH2sGSA/T+UWH?= =?us-ascii?Q?zkRBFK7yfg+ctABq5VXUMxV330tmU69BjzoME0nk4NT5BJBbVN+IT6eOvQtV?= =?us-ascii?Q?NKO++o6EESBWQ7ArTGp8c3JnlFs9njfrNTjWMtF3wWl5jm7UVTxYaS0SMdhW?= =?us-ascii?Q?i6nTFjnYGwRRcyBcLVMfyyDU9TKSKn4hsLs9wyB4yAn98/uY0OA+pT7jJVdi?= =?us-ascii?Q?EaQB2rzg0HNOkdNfUBN+iPcRB5aewY2KxEoQQUXjmTW4sf+trTIo7DN3TIC/?= =?us-ascii?Q?oiGa6bQoTBmLQNxP3fjSDR0XqXq+a/oVQehGjb7/Y4Hed4VF3c9B8kTFgmfJ?= =?us-ascii?Q?BxtbNI2o/lcV7rgSLuD0n32zfoupu2rJxJ51+lkCAqMj83VXFN37k0sO2ex9?= =?us-ascii?Q?p+M8wER3pzEkjuMUyMhZyJTxXw1CJs18cD6K33+Yv+VIFImMwkFbnc5BkK+i?= =?us-ascii?Q?Tpm2d/U3h+f9kZeFG2nGJZLionfS7F/e4LtNv2ZckBr9XULWoAZgUKK96Bke?= =?us-ascii?Q?HnJ9H4Pea4IdWNJLujijqelFzHR+JF/Akts1Kin1U2BdAZl1ah/BLmEE9WqP?= =?us-ascii?Q?Jzorz0ySIaP6g15ezpxOGe6tfk0ZedzjdSJnPQxCEZzH7NBzwSeF6oBu71dy?= =?us-ascii?Q?hTkYDpnP+6ZlEgSUIZ4oVPnuzDSSzxYf7qKpL5TU6dMj5+CKl2F29Jween+7?= =?us-ascii?Q?Cs3zBthm12eZIyOCdb2ugHpiMooLsD8jRcvsyMe3J0v5FVHFO/7lA7uklKL9?= =?us-ascii?Q?1/9RMnoLguCW5Sznc7fA0pOSl5X4u5/8I5ns8j4icQo2m89dWKvUqnd/TtLF?= =?us-ascii?Q?I5JWAWUWwZ3C//QATOsnjZ1e+WP6DA4AYib4CnwamEvsmnPeaUjatEXG7FWf?= =?us-ascii?Q?GGNCAbYl4NfcgVztEUYiEPa38zRdICG0RC3r4z+JCGIT/YxC4L/ZNX9lmz+G?= =?us-ascii?Q?xd9s0TIIrH0J5MywOKqQE7z5cmyTlh3D4PGKE8WZGBJRHLy+k+/25YWIHlDU?= =?us-ascii?Q?79IkCd6TkBAW2DtSPqvQO6dqXmX+s2nx7KUsz5TNztDV+21lSwhG8oxPyAGa?= =?us-ascii?Q?4p4PYJncVxdFVJ8h2tGBpfqKJ3UWs9X8WiZ3PkExXJ77U2IyY4TkKWoA+YVu?= =?us-ascii?Q?VdC4imGISMNyhy+etKk/z3hx67kb33HNkTz6JjHv2270BTQLR/e+fPqRCkC8?= =?us-ascii?Q?fqKXeg=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8cbc19d2-34d5-4326-214b-08dafd1b9b5e X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jan 2023 08:27:04.0959 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wSc4JGZwLJam9RJ807oiq7ihu3fB7G1/OoojRk4T9iA3tATg2qMkMCqYrmpJSxSw881JCngJjE+yymyFh9qSDtIwRzsHAueiR3jyWFhICJI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR10MB3642 X-TUID: /iSOxLpKrUA9 Am Sat, 14 Jan 2023 15:40:47 +0100 schrieb Baurzhan Ismagulov : > On Fri, Jan 13, 2023 at 06:14:01PM +0100, Henning Schild wrote: > > think we should not add a third pipeline. Too many switches and > > variations just mean that we will need testing for the tests and > > nobody will know which to run when. > > This is a very valid point. Our problem is, we need two (or maybe > more :( ) testsuites for efficient handling of maintenance tasks, and > they are very unsuitable for an average contributor. So this has > appeared out of necessity. > > I fully agree regarding the switches, I also hate them for the same > reason. We've considered moving to the raw avocado command line, but > it is neither short nor obvious, so we left ci_build.sh wrapper at > this stage. I would suggest a wrapper for every pipeline, rather than a switch to one of the wrappers. We can leave switches for advanced users but maybe only $@ to be passed to avocado. So a simple ls would tell one which tests are available, where the files hopefully have good names and the README or CONTRIBUTING files say which tests to best run after changing anything. Henning > > So, my proposal is to make "dev" the default and everyone should > normally run ci_build.sh without any switches (unless anyone > explicitly wants to experience some diversity in life). Of course, > the maintainers should then regularly run it. > > > > I did not try but i bet if "fast" would simply use Sstate caches it > > would run in less than 30min for small changes. And we would > > naturally test Sstate and actually become much better on "partial > > rebuild". > > > > So in my book "dev" is "fast" or even "full" with warm caches. > > Caching is also a good point. We can run "dev" with sstate by > default; we just need a way to specify local site configuration out > of git (e.g., in an environment variable). > > I wouldn't like to have "dev" = "full" + caching because we're > evaluating adding rebuild testcases with and without specific options > (like sstate, etc.) and running both in "full". > > "Fast" is rather a misnomer; its goal is to cover some 95+% of > functionality in somewhat reasonable time. The mentioned testcases > have been very consciously included there (e.g., sdk was broken > several times); we can discuss what to include but in general we'd > like to keep and increase the current coverage (execution time > permitting) without affecting the users who don't need it. > > "Dev" would further diverge from "fast" due to e.g.: > > * Omitted "pedantic / infrastructure" bits as you mentioned > > * Chosen rebuild testcases as this has often been broken in the past > > * To be useful, "dev" should be kept green as far as possible, with > all the implications. > > E.g., testcases can fail due to Debian issues, so "dev" shouldn't > include Debian-ports (riscv). But the maintainers need to "quickly" > assess the state of all supported arches. > > * Testcase granularity (one testcase covering one arch + distro) to > have a clear indication in the test results which specific combos > have failed and to be able to re-run that failed combo only. > > "Fast" needs to keep the multiconfig building to catch races. > > > With kind regards, > Baurzhan >