From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7091260455110311936 X-Received: by 2002:ac2:4f0d:0:b0:472:1b7d:2a93 with SMTP id k13-20020ac24f0d000000b004721b7d2a93mr5956645lfr.684.1651075188593; Wed, 27 Apr 2022 08:59:48 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:2815:b0:471:b373:9bb9 with SMTP id cf21-20020a056512281500b00471b3739bb9ls2338049lfb.3.gmail; Wed, 27 Apr 2022 08:59:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4txotKsPA4CUh9nTtZwA5XcAzaN40CTJp3yj7sedaD+IOOuB7Qs+Fy0R49cGHYfcufOiu X-Received: by 2002:a05:6512:a8b:b0:471:b8f5:b202 with SMTP id m11-20020a0565120a8b00b00471b8f5b202mr21017325lfu.31.1651075187353; Wed, 27 Apr 2022 08:59:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1651075187; cv=pass; d=google.com; s=arc-20160816; b=GoAcCEmAkewYzc6VDLSdWM3nzv49iQyqzIJevi/C80Vsa5/bKvt9ioIMM+fqFxc6jj hXTva0b84RlV3kVYH2VpUk6jD6vVrt2XMLtYEYMszYPUTBlcuWa34SczGBVAf8Sqg9Zw +z2lAG7Bd8Cfr6PuEEgJrk6GRi2651IF7dcz2wBsyAsrb+dYyw3C1jS5l0wejVFzOwgM q5QsJyXBMx3KJNjquV7chP/ByrVmAEV5xhJ06wZCQ98QgiUqMOS/Veujd0V64vasE44n yheC4l787Qx7pFQHMLyPN3q2Hq5HDP4OyvtzooyNqrJkxPhs56Xcycr5Ze0wHuy9Zihg f6cg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=wvx5ZxdO/Su1tHgRpYFLPU6kgCk1A9Tel/IVnpnIR44=; b=PhdDY0G1pqOkqB3WecZoAjqQnnAyurqzTMIynUR4Kq4+4tsQPRCWdssL6EZUQ0MSvn maQvt0vw0gIrqACp68+u/KxsPh6ocF7GssssOUz1MESbl1Q+/mEV5Fbs4i0DVTXamsO3 HijvV34r+pj5Kr1tL86OczIS4aqnCc3HinpNOjwoKzEiEngMBcpzXXvGkc1UIVBNqRKO 0N85LszWo7UPufxeX1lJQNJ8heBIgyJOY9aSAi02LR2ThHwcMgCmx7u7sECF+P1tDHne t3ivERpmtyrvspBqlmPl4kVCd81jzLypHTyFrzmQcWHM3Ug9/SNrT3LgQzrGgVLJG+DS dk9w== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=sGhoUqW7; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0d::601 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on0601.outbound.protection.outlook.com. [2a01:111:f400:fe0d::601]) by gmr-mx.google.com with ESMTPS id d22-20020a0565123d1600b0047223e9d7c6si87461lfv.4.2022.04.27.08.59.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Apr 2022 08:59:47 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0d::601 as permitted sender) client-ip=2a01:111:f400:fe0d::601; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=sGhoUqW7; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0d::601 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k5Pue2VPuFv7HQ5HaUtKSA1hTDGICHwTm07eXlyGty0UW1rCELq1JA0XN8kNyz8hYpc/S3tkjXkZhPXxJeWz9aj6imwuGG6eBaloNH+svFyjIv2g3K0JmvHiT+HeM3HOfIVE81lXzTkNoIyJ2P2sClIvtRYf+iP4oyagshd7JBImeM/zDRG4ZC7FYXU5ZzsOFS4vaOeWiITWMTAKEG2eyMg0fqVhr9pb/YCOAcKR/qWs8eqkhj9bktMya78iWPi/0jVOo0Q3OX/wPkHRmH7QhdIWn87TBv/XS7uELkfcLYmU20I1TlN2jTevvD2uRvqeLpUWJocpSi3bfawy6I/lHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wvx5ZxdO/Su1tHgRpYFLPU6kgCk1A9Tel/IVnpnIR44=; b=PD2ZQBZMZHE088aDuD7T3fZPlfNqoPFSb03KyV+Az7f6WUMQUdEaCeOSuAqC8iSOlKqULKle9I2RD7806/5qlumE7jmvtYlByNnOJtWVjZ5+5omyM5MrxPfDeZRQdgC79DzyC8dWX1xIHUEqqJIv3Tkkh9Q+AUhhsYR1wss1bWDw4I9jORgpqdDMsjOnE239gCwhFRoIIPWJCSIzRr86EhZ3HnJXZnaUH8qvZRlLDvR+Jv20CayBD1CzZicIR5t1WwRoRRG2TemxewP+8kJnrNoLpiVQMIXfsQAXDP2PfKsfF48V7usKy+1YhBeCdpOiMkjCxV7ch2ewQgytlYFK0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wvx5ZxdO/Su1tHgRpYFLPU6kgCk1A9Tel/IVnpnIR44=; b=sGhoUqW7UrqEzsI5oqrlcR6tLqoGZ+bCuosXIJ0w/DfGXLJPNzUiK8KTDaqpLSSV9/pDP0amRP7gPEW8GoJPDZXcxX9Yv44WfJwrYUKGmRtBcr6soZSWXUjD977YqX6ln2TitPPlsGbDB5qxEFYKlKZqGwbqjlcFkV0yZ21bB1c+ADuOpcUXSLtuy7PwV5lYfPtGWxq4TuTV4YmK8txWFLqY86ED47u0NZ/nGzLt1o9GNpw5jJ3dlG7/LMzY0jrI6EJlGMVHVwu17Or7NrpuxD6lqtgHgLOeRLAQWFph4GfnK0GxWbcPWEdqIZ2yJK1h+AXVCC8IjallDkrpQr1Rsg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by HE1PR1001MB1339.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:3:f5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Wed, 27 Apr 2022 15:59:45 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::f4be:8b5d:4314:c2d8]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::f4be:8b5d:4314:c2d8%4]) with mapi id 15.20.5186.021; Wed, 27 Apr 2022 15:59:45 +0000 Date: Wed, 27 Apr 2022 17:59:42 +0200 From: Henning Schild To: Anton Mikanovich Cc: isar-users@googlegroups.com Subject: Re: [PATCH 0/3] Make CI targets be configured Message-ID: <20220427175942.73ea0c28@md1za8fc.ad001.siemens.net> In-Reply-To: <20220427123205.30111-1-amikan@ilbers.de> References: <20220427123205.30111-1-amikan@ilbers.de> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.31; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: AS9PR06CA0052.eurprd06.prod.outlook.com (2603:10a6:20b:463::20) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d46a10cd-4ccb-46eb-5b80-08da2866f295 X-MS-TrafficTypeDiagnostic: HE1PR1001MB1339:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 35sMJnoa+0VpDW3c+8NTlUN/GQTplm6BjiCO3YbTbFtF+qiKVJn8Aamu3RMaKamdU88nU8BYQALBaPoPYDdYWxvFzNnaJCQ6pBuf2YahAviLIdh7PtRcEsqXq1xWKURd4Ps6Mk+eO3mPzVEmC7dT2vhEAc7zVKYwlrjQlhcW8odlx2YxkzTSfeFbAoUj5ZfRbbBhQZllcegOgvuWq9Wx1qjHHuJ2YU+ZhPSrKAsvydabjCUWF8P3CBd8Zvi90WaU60cx4OcuH01bCYsUx34F+hzuAevuKwwSLHfLSOsaECmMFCFDJwShD6dvcg3pyUwInqRvP9q6RmequXOj4Yf9bWXyI3xnHtuO/L11Tqfnm2NHCCaa/XlVNU6y3jeEy/LenDXEZw6W483O1vEKGeBCJ99ptaQ/gmeRKucuiJ6zFNxX2/2kJeDndOj0X+4o4WJcG0fpk0laz+OKZu3zoHjU12nIFZ2Hio1wrKiSvyY1TYbNuaeeUBvOiHdU0CxuaNPsKXCj7rQh4gzfD3JAbVbXqI5jjqifRp5kSccSgMCw2aaSZj7O2rTY5VAKHUDt3AlMiRCg/XYQDKNYwx/cA+waoNd9mqQyVXdZ9RvL0zM1JV1rIrSOtJYQMKUs/K/XGZ35lLK+NhPvpf3CFrdHQm/6pQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(6666004)(4326008)(508600001)(6512007)(6486002)(6506007)(186003)(86362001)(1076003)(66946007)(66476007)(83380400001)(66556008)(82960400001)(8676002)(38100700002)(5660300002)(8936002)(9686003)(6916009)(2906002)(316002)(44832011);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZV0ynExtfyz6XbT5aa43A+VhKdH+Ow3QzFJzXM55Hn4bU+D+TDSnyfW3yAdw?= =?us-ascii?Q?RDov56i3iE3Z6JjH9dAz7xqhRxI/Se0m39+cyy6FJtDD7XoThA5llXvk1087?= =?us-ascii?Q?QkajLK/o8R8MarHz9a1wUjbJx29WVtTwdq3/+2f+KZA5yzXplXPtqm1FQkG4?= =?us-ascii?Q?eD/UiuRiw+ScQZeMjdb/jIJhEh8ISglcyqr27Q3dj9aSsYL8g0c7rDzQiKOk?= =?us-ascii?Q?/K12Noz3YURsDL1KT0L5b+XSOKzkjl/7bumd2v5jzfbdJaBjm5xkw7giQFOM?= =?us-ascii?Q?yXEpbT4fJ3D4N8Jha+wFGnO7HJSPHzegoTJ6buRsIfbWSD32hBuKOhr3vuHT?= =?us-ascii?Q?eBqDknHS2xiXGQI44Y/qkFjzpHi9d4H4tHPyTXuQ6ClJ+ajuGJzda23G5w9f?= =?us-ascii?Q?pkMb3QWHAwNSIak4MwWcqzgQ5Rg7M8GEOQGtIfzB1Bwmr2mNOwdshxbRw8jA?= =?us-ascii?Q?GHt9M4u+Jm3DqUDPZkvRWQPtdW07QK7VKofQ9b+nFMc60T8LW3kG4Du+if8t?= =?us-ascii?Q?W9w+Cy2EWneUVD5iVm7gIODJjj++jXhjjTa3NHQJFta/mSWDUEFbh7lINGsw?= =?us-ascii?Q?IXdyLK6kFdGru/fRTcxRQdjTsvWL7Ji3A7aX8GBAUgPCy/Rm6TH41RlBwxk7?= =?us-ascii?Q?XRuVwGVqy+khbo78zIQoNs9rY3xQyInuNqNIfbbD3ZrolDB75F0OyiRb4mBe?= =?us-ascii?Q?5zYvSRCw8O2knbnVHZKaGI5tVM+5X+F4p73qgh9t7DjE50Kzzr1nXabmGEUx?= =?us-ascii?Q?wcWf72XtgnT7Q2NS+BBq777WivKhmONWD8PKDpimaPkS6oyHuESrztbKVhf2?= =?us-ascii?Q?Wyqgnzjx3yJGhf/yatWL5sgJPf+2FjedpWfsaN5tCb1640DBDtDjKPdWo1bA?= =?us-ascii?Q?yXaybfe1kfKoQiJ43bqUBs1EDfl72eFW4yNyLcsK10O4Sd+j7rH7b6eyA+vn?= =?us-ascii?Q?CllZCVDBeCSHBrKNUxJQZE1zjhEZ6bMujpxog8piRYMj66zqoGKVSFTRSyd7?= =?us-ascii?Q?zrPrftQHp+b6pBFNY1J+YbWEHZn9PVWWMWGY9Ck8Cc8bRwVihJFfgNuVLQ9F?= =?us-ascii?Q?zRFALcniMoz0eX4EACUQ1YXtIbYet30M5d8Mqp9oO5hrLdVCMYPJ7Q8CWkXd?= =?us-ascii?Q?Cxhwa0RoPJWnaG7MybaKAu8gr1s4EM0NSOwfCogR2JMYpaSJKHegbUX8vxHV?= =?us-ascii?Q?lfSf5P8QQh02uIud4OBsGy1CiY4yxoKVik4QMSP5xjXejs5nc8eTOFBnqE2G?= =?us-ascii?Q?gcePWYPimqRgJHawnZQE/t8k2fE6bYcl4diAMVapcWuFvNqes79fZpP10Yhh?= =?us-ascii?Q?md2ZfzMq+1dkWoUYbUyBiBn6xaQsvbO6v9LoDpFEe9rwf1VTH4XvzfeSsGTg?= =?us-ascii?Q?jRFIMGcZ6b5W9Q75nW1LrqXPbhX9/3+Iu1hwyAXYlJ+v98DObfB5HB8RajBz?= =?us-ascii?Q?6DS2VmhIP3TPSM6Qa96Cp7LGxzsw14b8DJ0CxEpvG155OjCaKuWzdcv1M9tw?= =?us-ascii?Q?YJX09QDO2dT6HstIdvivOLUSwC2bbXv4CXG2ZsNlKBznMhF2MLtAANL/SdiF?= =?us-ascii?Q?FuNWBvSkRC27ORKfZ3Q2Y5JR7bA1JYgvk+RgqrmPpm/wL/HbqhEGV6FSfJQM?= =?us-ascii?Q?RDFQcNNSVvbir+p00e4PRPrViPKiVO2Y995lGUu4YG4D5/UELmsvGZYYFqiU?= =?us-ascii?Q?Tdr0Z3yRIym0o6UCAPA5zWvJgcpyZtapbJM0XCAZbOqj/tN1MAoRALCCDibc?= =?us-ascii?Q?sFS+2UcGNzB6vHwwiW5V5CCbmqnscDSSDZO3cXp4m+RidtPV7NY3M4Db3ckw?= X-MS-Exchange-AntiSpam-MessageData-1: o+kpirkIFRRGPfGLxYaG8Jk9gtwhhSruZN4= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: d46a10cd-4ccb-46eb-5b80-08da2866f295 X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2022 15:59:44.9715 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: J5pzDsYWIn2DU6YYO7AkkBtEPacWY0bVGm7B5V15cTMNImQyyvZqsECSQEYdLCJ76owzGafk0jfP3/xoYURnEC1BTiMxYr8xl5JXJGM/zjQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR1001MB1339 X-TUID: tULM/zUqp3FT The test suite already has too many knobs, now including env variables would be the number one cause of "works for me" ... does not work in CI-A and even different in CI-B. I think if i pushed this series to our CI it would simply fail, because i would not have any of the variables set there ... and no clue how you did set them on your jenkins. Tests that are known to not work should probably be skipped in general, not kfailed because that would just waste time. If we ever see certain tests not working in certain setups we can see how we make skipping configurable, but it is imho a very bad idea to introduce that without a need. So i would say let us get rid of some of the KFAILs and turn the still valid ones into SKIPs ... and see about the rest laster. Henning Am Wed, 27 Apr 2022 15:32:02 +0300 schrieb Anton Mikanovich : > This patchset removes hardcoded KFAIL protection from targets and > makes any target be KFAILed or SKIPed based on execution environment. > ISAR_CI_KFAIL and ISAR_CI_SKIP variables are used to store those > lists. > > Anton Mikanovich (3): > ci: Implement dynamic KFAIL checking > ci: Implement dynamic tests skipping > ci: Correct container test case name > > testsuite/cibuilder.py | 56 > ++++++++++++++++++++++++++++-------------- testsuite/citest.py | > 32 ++++++------------------ 2 files changed, 45 insertions(+), 43 > deletions(-) >