From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7186996775974076416 X-Received: by 2002:a9f:3525:0:b0:418:c0a0:556d with SMTP id o34-20020a9f3525000000b00418c0a0556dmr9932301uao.104.1673707252241; Sat, 14 Jan 2023 06:40:52 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6102:2386:b0:3b0:9714:6c49 with SMTP id v6-20020a056102238600b003b097146c49ls2748739vsr.3.-pod-prod-gmail; Sat, 14 Jan 2023 06:40:51 -0800 (PST) X-Google-Smtp-Source: AMrXdXt+cmnFrqmEmyuYiJU4rxKcT+II0XrixkPLOEk4m86ZqNLS4Kv0KMpDctkFBy6EtMiy4OTG X-Received: by 2002:a67:de19:0:b0:3cb:db7:3ac0 with SMTP id q25-20020a67de19000000b003cb0db73ac0mr36924943vsk.30.1673707251590; Sat, 14 Jan 2023 06:40:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673707251; cv=none; d=google.com; s=arc-20160816; b=M+WMn+R+YYma+gx/0Jx0ZIO+qBRV4WE04Q8prJJvAwSmi3Q6ap2vMBBwDZdAg+VZSp jK5YwXmFfzaU7HmwOHzz66KyfBmDwPp85rMh7kSx9IcrxOpMBRAqA9YRqHLciOnE696j 5uqqoAYzKjqWPxBV3I5ws74xahk+rtS+t9Puq3t0Om0EGfUUNj7XtWr4YNQKubn9g6+g 2f3bLSR7zqraB2aNttnYYavfWunR0l3Pg6DOL3g6qTU+3ldnXtKU/dOKCDIicexGz9dG Ugqa+6WzIKDUX9LW7JT4rM++alzmQ7AxJGF/ZhsS/OblnYqwS3LDHld5X2leJsF29E7e 5ZXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date; bh=eWU0h6mVYDWjVeBgi1Vw+vj+3mkdsgVPVMSVaVIf+po=; b=uHPxqSLn3whGHjEFd6WeB2W6RWKennLofi3o++Tab+Y2PFcbO++GxC9lYMP2mNnRnV ORI+MIuyPe3fABEjrkn/j+fo1HFLVBWGlYAAZ6R5FA4sot6FCNzHhNjMIZcCi8C2BSiy KeUtV0puoj8dxlfZ+7aqbL3pJi7uivQ/sTiNTzFgJqVj562TVW8AUCRo0zWVZ8y/3l94 eHaAqaYuX/YUuIpogwqj+CB72Ki04BRHf+/8vadidzLT0EkFj4B02uhLDbapnxssERsY mUWATdUX2QGWg6lH9xitumYEb/6AsWFPAbZVnK57LCmxPATiAsP3h5Rn6NomAF94pke6 T5Pg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id b15-20020a67e98f000000b003980b6c8861si1628642vso.2.2023.01.14.06.40.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 14 Jan 2023 06:40:51 -0800 (PST) Received-SPF: neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Received: from ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 30EEelvp001891 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 14 Jan 2023 15:40:49 +0100 Date: Sat, 14 Jan 2023 15:40:47 +0100 From: Baurzhan Ismagulov To: isar-users@googlegroups.com Cc: Henning Schild , "Moessbauer, Felix" Subject: Re: [PATCH 0/8] CI rework of gitlab fast job Message-ID: Mail-Followup-To: isar-users@googlegroups.com, Henning Schild , "Moessbauer, Felix" References: <20230110121748.14917-1-henning.schild@siemens.com> <20230112161250.78000195@md1za8fc.ad001.siemens.net> <20230113004315.65f968dc@md1za8fc.ad001.siemens.net> <20230113181401.3b8e5807@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230113181401.3b8e5807@md1za8fc.ad001.siemens.net> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: UkDg7vtoDYW4 On Fri, Jan 13, 2023 at 06:14:01PM +0100, Henning Schild wrote: > think we should not add a third pipeline. Too many switches and > variations just mean that we will need testing for the tests and nobody > will know which to run when. This is a very valid point. Our problem is, we need two (or maybe more :( ) testsuites for efficient handling of maintenance tasks, and they are very unsuitable for an average contributor. So this has appeared out of necessity. I fully agree regarding the switches, I also hate them for the same reason. We've considered moving to the raw avocado command line, but it is neither short nor obvious, so we left ci_build.sh wrapper at this stage. So, my proposal is to make "dev" the default and everyone should normally run ci_build.sh without any switches (unless anyone explicitly wants to experience some diversity in life). Of course, the maintainers should then regularly run it. > I did not try but i bet if "fast" would simply use Sstate caches it > would run in less than 30min for small changes. And we would naturally > test Sstate and actually become much better on "partial rebuild". > > So in my book "dev" is "fast" or even "full" with warm caches. Caching is also a good point. We can run "dev" with sstate by default; we just need a way to specify local site configuration out of git (e.g., in an environment variable). I wouldn't like to have "dev" = "full" + caching because we're evaluating adding rebuild testcases with and without specific options (like sstate, etc.) and running both in "full". "Fast" is rather a misnomer; its goal is to cover some 95+% of functionality in somewhat reasonable time. The mentioned testcases have been very consciously included there (e.g., sdk was broken several times); we can discuss what to include but in general we'd like to keep and increase the current coverage (execution time permitting) without affecting the users who don't need it. "Dev" would further diverge from "fast" due to e.g.: * Omitted "pedantic / infrastructure" bits as you mentioned * Chosen rebuild testcases as this has often been broken in the past * To be useful, "dev" should be kept green as far as possible, with all the implications. E.g., testcases can fail due to Debian issues, so "dev" shouldn't include Debian-ports (riscv). But the maintainers need to "quickly" assess the state of all supported arches. * Testcase granularity (one testcase covering one arch + distro) to have a clear indication in the test results which specific combos have failed and to be able to re-run that failed combo only. "Fast" needs to keep the multiconfig building to catch races. With kind regards, Baurzhan