From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6693749539433611264 X-Received: by 2002:a2e:9d09:: with SMTP id t9mr12869728lji.151.1558554260109; Wed, 22 May 2019 12:44:20 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:c106:: with SMTP id r6ls346786lff.3.gmail; Wed, 22 May 2019 12:44:19 -0700 (PDT) X-Google-Smtp-Source: APXvYqxVn9ovuUfbkYx/FBFkzEViAsmdRxxXQFnElh6SKKC3oS8WjseWbqBPQQm6kH+7K4aJziAJ X-Received: by 2002:a19:1dc3:: with SMTP id d186mr43856994lfd.101.1558554259649; Wed, 22 May 2019 12:44:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558554259; cv=none; d=google.com; s=arc-20160816; b=aXGleftRIS/m8dYdnsflHlUghFrOg7hv+9oyEco1T+Qxd7rUYxwEfCvblnM8gM6t5t Rx/nL5BzFpTLN3uYGZy5adwrQs5+tneIVRwYHLJq69O96E/ZWJrzuwlLT+eZrDQr/+zc i3gib7ZvZDiH3KOHt0NN5tIh++zmVo1jX9+mCfmOTjJh6k2sa0NPPOBSBKM7pG3EX1kl TCU+Gc4jKaB9GGlX5dwDi7YDOA0gkKeSJpILkKHYhwau9IJ82IARtEQjLyYI6DBn1WQm WocVnRb6txr8qw3YdwLbbMvkAMLzsjk+GNi5tttX79CJuGLVsIhmeN+esR0s7EGVcqmi 8mYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:to:from:date; bh=b49IRg9cc2DbnbHQGF97pKx2XXXsb8oIKLRJT9lnfqY=; b=Ff3/jlUUrf2t76vvLSy5lo2VRHhvAQErrut1pfFliePsEZ4hUHw015pC4AcJ1Gi07J d84p99aBqQb0fNQ+gfuv3lPQRsvrfsBdf9ARCiIPmh1A9kzycAv67jaxKwREmYjsSJiU 5NEkBlIOe0k2x3N0WhgUeX3VPzuyJMts4ODMobrDsERc7iNU/N8qlvIWpU8UpltukyFO 66vLhV+C3LG0cWkhIcQjOCuoQZ+Ojnyg6VfvPHpqHGHrektmJn0TyKtgJGbBQy0aEUhb 0a/t3T0oXCRe0xPvJy9iLZ0TaWpFSHkr53uJjqcdRF6/ElYQ8+jRzZrtvIMucpEcb3Ou EADA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id h3si1899886lfm.3.2019.05.22.12.44.19 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 22 May 2019 12:44:19 -0700 (PDT) Received-SPF: neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Received: from yssyq.m.ilbers.de (dslb-002-200-058-019.002.200.pools.vodafone-ip.de [2.200.58.19]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x4MJiGg8006204 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 22 May 2019 21:44:17 +0200 Date: Wed, 22 May 2019 21:44:11 +0200 From: Baurzhan Ismagulov To: isar-users Subject: Re: [DISCUSSIONS] CI build Message-ID: <20190522194410.GC2441@yssyq.m.ilbers.de> Mail-Followup-To: isar-users References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: +ycv1n6saEK6 Hi Claudius, thanks for summarizing. I think we'll open github issues at least for some of the items. On Wed, May 22, 2019 at 09:26:51AM +0200, Claudius Heine wrote: > Here are my issues of the current CI with Jenkins in text: > - Unreliable build: > -> Network and other unrelated issues to the patch > -> Contains expected to fail tests We'll work on these. > - Not transparent configuration/environment: > -> parameters with which ci_build.sh is called are not controllable from > the repository Hmm, the intention actually was that everyone runs the same fixed testsuites. If desired, we could publish the arguments and gradually move them to the isar repo. > -> difficult to select which test case should be run This is the motivation for Avocado. > -> Configuration of the environment is not controllable from the > repository: Installed packages, System configuration (locales), > Distribution, etc. Here the intention has also been to have one standard testing environment, and it is Debian stable + instructions from Quick Start, nothing more. I'd welcome moving to kas + docker. If desired, we could publish the current setup. I'd like to have the testsuite be runnable from any CI system with the same script, without arguments. Then people could deploy any system they like (Jenkins, GitLab, Travis CI, AWS...). The same testsuite should also be runnable on one's host (already the case with ci_build, vm_smoke_test). Under the hood, every test case should be defined and runnable individually (Avocado). > - Just one giant build process > -> Takes a long time This will be solved with Avocado. > -> Cannot be spread over multiple servers Haven't looked at that, ideas welcome. Jan told us he is preparing something, but load balancing would spread multiple pipelines, not tasks within one pipeline. Depending on the costs, it might or might not be cheaper for us to throw more physical servers on the task. > -> Giant log file, that makes it difficult to figure out exactly which > command or test case caused an issue to reproduce that locally This will be solved with Avocado. > - Manually triggering of build jobs > -> After pushing of CI branch, build has to be triggered via button I actually prefer working in this mode. Henning has polling enabled. If you prefer two branches, please send the second branch URL to Maxim. > -> Patches on ML should be automatically tested and reported back to ML > (that only makes sense if the build is reliable though) I haven't looked at that. Is there a tool that does this? > - No "test everything" (acceptance test) build job Currently, it's the combination of cross and native builds. However, there are a couple of steps that are still performed manually for various reasons. I'd like to see them automated if at all possible. > - Logs aren't public accessible Logs are accessible with authentication. Jenkins has public mode, but it lists the users publicly, that is why we haven't opened it till now. If desired, we could look at that. Is it that high-priority, given that all major contributors here have access? With kind regards, Baurzhan.