From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7117224748438781952 X-Received: by 2002:a05:6512:3081:b0:481:1691:e7ad with SMTP id z1-20020a056512308100b004811691e7admr32020136lfd.396.1657186302939; Thu, 07 Jul 2022 02:31:42 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:99c6:0:b0:25a:89ff:c693 with SMTP id l6-20020a2e99c6000000b0025a89ffc693ls7176878ljj.9.gmail; Thu, 07 Jul 2022 02:31:41 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tZE9x3UVj9IB1ByQzWnWlZ899sCxdlvJON4CAghbsdX2Vr8o8SYhXWKzW4Q6OhcvGXDL3Q X-Received: by 2002:a2e:7c1a:0:b0:25b:c025:c344 with SMTP id x26-20020a2e7c1a000000b0025bc025c344mr26929509ljc.322.1657186301852; Thu, 07 Jul 2022 02:31:41 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1657186301; cv=pass; d=google.com; s=arc-20160816; b=OsLVi5FuJZiIShiu+ZEl/IvjFkU07esyez2QTzm6GSc09Yw1C6N1M6fZaxSB1aVuFO bwtcWDRdt40hDQinQV7oNJkOlnTMeIhwuSoM+kIhMqP61inPdLLCa/C+fiqENeu+qWFq BdYn9sD9/uCT/iCzmDEXxPJvpOT3b0BQientdWf/weBHaNQ0ODvrIGGvPA0PQcKxhfbg 2mi6E/i5rUqdI2hBnFqPT9Vq0MHvtB21YWVnZUxnRWne8MQwagwmYUczfJocxGYYG3P2 7cd87+LRSXxGKrNLf9b36VGfbh1SdFQalG39QlEzG54c2FJkkcvztvFzBpl+sMm4VPb5 D2cA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=Uc6MWwBBxDtFVRq0Sg1dK3U2AFnok9L3yXs0+VYNPlo=; b=LQXksTPCp+HLc2V2Pa2qpYec8Z05RMWMd7nrddhq2gLuNfTa0cEPIPja04ppaAr2tC BXMTIVSSndUCvQE12Ik/oOFU7OUIhEJb5NLhAWQeKFvRznf+KJzNa4Xa4XI6CbvjFZNO BRXChqaAdEa/qtXD7O6jmSVNW0CTF7ehFzKTSgTIN+T51DnLbaon1HgNREr3mdMObYG0 sWOzmUopnDx9kCYHCuMxi8zFzSDXH2kgPz0CZix62JnNm4Lz8/Q+ggPbbAr2zGAz5zn5 hVyWGSRZLwi0hErXi6VHPkwRpp+p3FTABB9GiDypIYF/fw9WqF7ZP+VOdWiImwcPZd18 sYBw== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b="Wl3kr/zH"; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.21.74 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2074.outbound.protection.outlook.com. [40.107.21.74]) by gmr-mx.google.com with ESMTPS id k27-20020a2ea27b000000b0025d2c310ccesi344181ljm.2.2022.07.07.02.31.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Jul 2022 02:31:41 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.21.74 as permitted sender) client-ip=40.107.21.74; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b="Wl3kr/zH"; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.21.74 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RvfZBauZSfmnH3axntFm9OmuSsN9vijJYcX9MTCTvEuZRoiHxwGUqczKAsyc2BKKQwQgmVfyiXCZNCdzE+3Fo9TjFGut4yI4sNi/RvG74huWREw9wNzsvsXY7JdGTJZJCJhtVXcCgm8DLIYb70YgSlbrg1HZheASwP1vz30bc8fnqah/5qWhOu1nqhcXDz97S3tpzF0cW6FO2j393NifpsJzKnYhSVHXlr2xTcO9IsxQqKonYd3teQLpaQZNp/yGMUgB0rAL/7aHjOEYWEBQaP4Sq9ULqXHBe8o7YqMuGK/EmZrmVRgFsB/7RKV4q+hNXvIbRLh8RgsK7zBEsSkMrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Uc6MWwBBxDtFVRq0Sg1dK3U2AFnok9L3yXs0+VYNPlo=; b=FzwwmckXsHL42kSu8jbCpuPi+yPbx0X3EL5AvbMsjgSIn7FsBh3Iww9FikDYcOe7OYUTHT44xB6BIGNSepVVC7F3ZrL0o5ea+35tbc3gX/Ochzaa/Y9oF5/fn0LQre8t1Ks3H9TgqlIz+GfIIyH4rlLf6pIu7zknSehLCWu/abUVyuG54BsQYxO5XjhPvZkZ+0tVAbvOunMzEGall/0Av0rcRE5ayzaPxRqrgsaZq1lsHSYyVZyrZlK2tw2Vd57t/gHN+/HcZitLb6Ef+T7kYkDQMFugL9mhFWlcCjPs5piiH4Nm3bL8U+9FjPehP3JkoiC9sFSSXUzesDxb12sewg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Uc6MWwBBxDtFVRq0Sg1dK3U2AFnok9L3yXs0+VYNPlo=; b=Wl3kr/zHMsiBhcSaSIdx+WfJ4W7BYEoWh1SwZpLcIwO/oLLwvTc5S/4euLUX7aLV0vl+aI8M7rdxpbWM4GvAApUYdxu30BhOfmXke0PweWyvzRighkxWhw6DJUejeaNXu0DUB5G4/Kk8a2D9waz6BsxAYhKP8+QvchK+FvVZ+lb9uU2dgibh+qGvbKfKyvCVd64vAWyZDuQI9rirS4suK+zRH6FZD2HGl/tMe+u4zQVcS8l8JAYoPYE8lxgAwSxanhRCCCFNUMtRJih2E0of369nkGkAAGGPG7wymdP67HZO/MDraxu9R0FyhwUo2bGymhrVi61Q26exttdl6omptQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by HE1PR1001MB1084.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:3:68::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.15; Thu, 7 Jul 2022 09:31:39 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::b925:3d3d:8f5f:c4d9]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::b925:3d3d:8f5f:c4d9%9]) with mapi id 15.20.5395.021; Thu, 7 Jul 2022 09:31:38 +0000 Date: Thu, 7 Jul 2022 11:31:36 +0200 From: Henning Schild To: Baurzhan Ismagulov Cc: isar-users@googlegroups.com, Quirin Gylstorff Subject: Re: [PATCH 2/2] scripts/ci_build.sh: Set avocado-framework version to 96.0 Message-ID: <20220707113136.12c89e2d@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20220706114655.119026-1-Quirin.Gylstorff@siemens.com> <20220706114655.119026-3-Quirin.Gylstorff@siemens.com> <20220706160148.12e8c7ab@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.34; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS9PR06CA0067.eurprd06.prod.outlook.com (2603:10a6:20b:464::26) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a8889cdf-8b8e-4097-0232-08da5ffb7e5c X-MS-TrafficTypeDiagnostic: HE1PR1001MB1084:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9QSkhKC3S85ytWfPclKN5ZiF4NTDP+AGSfGvd6FTcoZjY+WkEedqUySAoVTJJyPdmrDhg5YMrU9xlEyzxq6JMZxqU1Lt0/meMhcpZlHRrKdPfuNHpw5ITnS+Vts7xmpSZK4q7h+2Cb0+xREHqIY56QcnAlsNhoF6sdpXzS/w5HaVYVxC99vxxdKZZxa8/w9tjbt2VbUBzUphpWawGmtZnmw7JgcRuEdOFKiRSGYQH0n7HoCZDdVWHrZJhlnQ9D9U6bjdLyCpX8FBCYJW+KhJQ148ReZ3f4vXtIPTHGPLxTqVqPIBVd43QOe8mdUBy6Fr2Psgqx6564KN4njawp6bVqSJ5cW3/ZMuQZVvlAoTsfjSNXzKxczUzu7JHEBDyqAQhlHwA0ud4lx3kaVfByHuUutpBsufoSjHMACyJALFTljUjNUH+4C7d74SVmffaPnNfY640Q8rito6eCCsQ8zw/3ehfJ6+8QzApJUWFM5IM7Odc4WFYXYUDmvTLTLhSTLBH+tuqgo0J4QM1pa0bvJk8hIHvXiYK5x1bWToLSPOyh3jSmWQ2IMKhpSWpuFRy7bCa1MhFEVSkiZB1EyJdNh/KhUyO2wL2lDE4CiYIDOal3DOWiLDUs/ZxGdL990+LxUSYtDGbhP62w8anFCp45luvigkJfzUj4CxvpZTu8ardWSm5P81EGVk6doGve1TOZL33b2lb0eE68C7DmthhfjPXYvw6YmiPvf+RBdaVOBrNexMu1KBo03etTy5i5Zr8uHF1cr3dNlnzwKXzqyTQp8N2Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(39860400002)(396003)(366004)(346002)(136003)(83380400001)(41300700001)(44832011)(8936002)(86362001)(6916009)(5660300002)(2906002)(66556008)(38100700002)(66946007)(186003)(66476007)(8676002)(4326008)(6506007)(9686003)(1076003)(478600001)(6486002)(107886003)(316002)(82960400001)(26005)(6512007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?oLgGBHkO/XHerimG+sFE82BiHFgkeS7ZwHoN8gcE3HRvfifExgtxYfxnxPEi?= =?us-ascii?Q?vbGuXKFj3sZmp66B/R09N1SzHONwRLLrcoO34Y81hwSa/0iSLMtEBsBX2TVl?= =?us-ascii?Q?XJa/DvwBc1LO7pkEbOXP1e8F0jLaJzIH7JJ0W6QUY0mDK04Pf8DysP5X4EAq?= =?us-ascii?Q?5arWiW8D3Q3BDoQsHDBRWqJYkWMP8k9azjE6Ch7DNCW1wJBBXaznfK00xPx9?= =?us-ascii?Q?No+A4MpF58BEQmUrZHHPaN77YMnl4DXo/E3k3bISZ1gig4GO8PPoCQxZUQ4K?= =?us-ascii?Q?AI5SltI0pTNPKZVlC2PEDP8FtyKKRKFhcO0nBTTftqylELM7SAaMFVE0laxH?= =?us-ascii?Q?KFumjzWQQI/piywcGv7urP8ObsnQnIRMuZh+g5PDTRdrogNozsB277mFGUCj?= =?us-ascii?Q?qu70/yuCNZ0K9mZq8F01lULY7+wawLvymZ0R2wyi9S+xjKlQJW1tFwuDyXIk?= =?us-ascii?Q?Iinztg/9F6JoxuCvkqI5PPV4gU66YQiaaOasEYpvXvrJEAqVYRIAbOmidV94?= =?us-ascii?Q?7xPV81t4i/sv/XyEcJUgiWMjWgESOcmGETOvGx5xd1JXmtlqRbKCI155Zf8p?= =?us-ascii?Q?9O5C7pf3BubhzP9CZdYSUvE9g/NQonWtLMV/zHSnWJlSocvsLImeZ8KCCwt7?= =?us-ascii?Q?RSuVPFqHbvY5VLctuO5iE5qT9srA/MmprMt7apIl4mGbxYCVN2V1xps5/H+p?= =?us-ascii?Q?xx8hOku0b49L3GD9cmOjQgg7Ir9Yv3L5bt2nvzN6AEESl+LMx64clo2ENNLp?= =?us-ascii?Q?7Y1gZZrupApmAgXtcBf7IfAg1g4aq0EVevaUVL+T6xaT34qe/CVyMGra48Vk?= =?us-ascii?Q?DwTvJO4EoL34U5kedbK28q5jp0TF0OnoOrhvrz1IEtnPTwdalEUmQrY+e3cl?= =?us-ascii?Q?Up5GbDwz/EXBfiTREh9wBCPeuqy08NVDipvIma9Nle+4iSKJQqLmCui6G2/I?= =?us-ascii?Q?KxbJVPfVJ+qjs2E0MZJkKk/FO9LwJt8md2YdkoYm3z/mxH2FrRPz0XidoPoj?= =?us-ascii?Q?KUpatYIfIhxSllIbuSBBg/teiEAclEJs7c8px2rRU8DEuUCDMUcQn+qnlAB7?= =?us-ascii?Q?ZY8gBVTzH+wxjc//qVdW3PrCheG0FglqPp2kh2643zjcwDHrik3YbkXqu7A0?= =?us-ascii?Q?j4WcJqYiBrmIIXbrMJvdsSeshJDm28vBLI+3oWOUVS19eMR0x8MxgPCbS5tL?= =?us-ascii?Q?ajTb56jngVaiTUh0YbhXP7uNMQav+5DX5AFbhzDoXGV8lOH9y59WMd7MSz9w?= =?us-ascii?Q?KbjyZngrIyDHOtFbs+a6o4/6yVRddeKt1eAg8csqffDcfSGYMCIzFnGiAUJP?= =?us-ascii?Q?oWfFh+b0BshTkjiUQsBtUjdz221RSmkStLwX65bT9eXvZRxHFdZ+RcpD1P9/?= =?us-ascii?Q?BWakYBBBZc2zFJOVfDBeGcdBASnGOOTpsgXLCn9xCuKrm3WFUWGNHjNQpxWd?= =?us-ascii?Q?xjOxEE7lGwivad/DXy4m8JPyekzvSRKCXnXM/MQoKmJ0MrDWBEeYgcMzo4rX?= =?us-ascii?Q?RIejqmn53jtCGVQDUtphNIygY7WZimHe1QjxBPIp/aYne5+OOWMSglm4avZm?= =?us-ascii?Q?sm5BhH6Xxy6tSZUcLW30eur90CAd6FmEo2eid4f9L/MB4axNyPRmS78u/A68?= =?us-ascii?Q?Ag=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: a8889cdf-8b8e-4097-0232-08da5ffb7e5c X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jul 2022 09:31:38.8672 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dvRwlQEHA/UpGVFJQh+XFFT3YX4vVPr9UblfuiMrkKdezceb4ghBi3J9nmeUPmMZdOMi6E98kDfdjecwYUwgdqfceM/F2fQLWSg4uZPqC60= X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR1001MB1084 X-TUID: Tf8hjzQmTxtD Am Wed, 6 Jul 2022 18:27:41 +0200 schrieb Baurzhan Ismagulov : > On Wed, Jul 06, 2022 at 04:01:48PM +0200, Henning Schild wrote: > > > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh > > > index 2513f1a0..b1ec6912 100755 > > > --- a/scripts/ci_build.sh > > > +++ b/scripts/ci_build.sh > > > @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then > > > rm -rf /tmp/avocado_venv > > > virtualenv --python python3 /tmp/avocado_venv > > > source /tmp/avocado_venv/bin/activate > > > - pip install avocado-framework > > > + pip install avocado-framework=3D=3D96.0 =20 > >=20 > > Once the code is reworked so it does not use the legacy feature any > > longer this pinning should probably not be dropped, it can be moved > > every once in a while, but getting a random version is simply not > > good. As we see again here. =20 >=20 > Not including avocado in kas-container is kas's decision. This > workaround was requested on this list and we applied it. If you ask > me, I don't want to have it and still suggest to use debs. pip is kind of ok when pinning the version, removing the pin was the problem >=20 > > Especially not when ilbers does not run this bit in their CI, having > > avocado already installed in some version. Maybe we should always > > use that venv/pip version no matter if the host is contaminated > > already with some other random version. =20 >=20 > This is one of the reasons to avoid pip, which we have repeatedly > suggested. That deb of yours does not count as a deb. It is as unmaintained as the debianization in avocado itself, probably not tested in bookworm ... and what have you. The pip way is at least maintained and recent. If one never updates avocado one will happily write tests using deprecated features ... forcing oneself to stick to the old version forever. Not to mention that some people might want to run stuff on non-debian distros. i.e. the one i tried to get in with the arm64 efi booting, the build itself done in a container (debian) the boot tests initially done/developed on the host (gentoo). >=20 > > How is the mainline issue for debian going anyhow? =20 >=20 > We've contacted Debian, got initial feedback and fixed most of that. > The remaining issues will be clarified with the upstream. We'll send > the updated version and ask for the review. >=20 >=20 > > Time to grab some tequila to wash down the guacamole and sing some > > "i told you so" songs. pip pip hurray! =20 >=20 > I call everyone on this list to report the issues and discuss specific > solutions. I can also start "told you to use debs" songs which won't > solve anyone's problems. Both lack debian upstream quality. Again it is fine as long as it has the =3D=3D ... maybe together with a --no-deps if possible. >=20 > > When it comes to testing the maintainer should really make it as > > simple as possible for people to run the tests, ideally with the > > very same outcome as the pipelines of the maintainer. =20 >=20 > I think providing a deb in an apt repo is as simple as possible for > people to run the tests. Way simpler and cleaner than the pip stuff. > And using the same deb would result in the same outcome. I do not think so. What one could do is fetch upstream avocado, build their package and install it. Do that every single time, also in the jenkins setups.=20 > > Also it should be easy for > > users to contribute to the test suite, so it will be reasonable to > > ask people to bring tests with their features. > > IMHO the whole avocado story made both aspects worse than before. =20 >=20 > If you have specific difficulties contributing to the testsuite, feel > free to share them here. From the frameworks that you had suggested, > any of those would require getting familiar with writing testcases, > so currently I don't see major differences in this specific aspect, > but there are important advantages in other aspects. I admit that > since we spend huge amounts of machine-hours for running CI on every > patch set, we do have special requirements for the test framework. > And I think the result is good for downstreams as well. >>From what i saw the tests are pretty convoluted and python is not properly used. But that has nothing to do with avocado, just the way the tests have been written. > > Only > > for some features that the maintainer wanted, but what they get is > > ontested patches because they are the only ones able to run that > > beast. =20 >=20 > This is not true, you are running avocado in your CI. You can also > run specific testcases in kas-container, just like with any other > framework. If desired, we could contribute docs or make a community > workshop on using the testsuite. We are trying to ... sometimes ... if it works. But for sure not always. I am still often finding myself in the place where i send untested stuff ... well local manual tested and then send. And one reason people skip tests is still that it is pretty hard and time consuming to run them. And they are very random and likely to fail anyhow. So i just send ... maybe i push to my poller branch of your jenkins ... but this will always go red anyhow ... like next does all the time. The testsuite failing in more than 20% in the cases anyways ... means people stop caring about it. It is "rainy thunderstorm" in jenkins anyhow or "red" in gitlab ... it was before your patch and it remains with your patch. At least the reports ease that a bit, if people still care to look what is behind "red" or "thunderstorm" Henning >=20 > With kind regards, > Baurzhan