From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6693749539433611264 X-Received: by 2002:a1c:1d46:: with SMTP id d67mr9516387wmd.98.1558586653634; Wed, 22 May 2019 21:44:13 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:6141:: with SMTP id v62ls1303571wmb.5.canary-gmail; Wed, 22 May 2019 21:44:13 -0700 (PDT) X-Google-Smtp-Source: APXvYqyG2lLaw5ceitedxprJmQ4xhWc1OsIZepjk/gKuPqz7lO9P0UFvTff/WRV3lzJBTvVuudMn X-Received: by 2002:a1c:680a:: with SMTP id d10mr10220836wmc.145.1558586653121; Wed, 22 May 2019 21:44:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558586653; cv=none; d=google.com; s=arc-20160816; b=bYtCU6yiCcjePr8W9Lh29Bg0IIThOtcfOED87ueK5z0Qfax0gqv1L+DmaqZIDdcDhS stEb3MH2snnalfjt0KYl/ql9foIe5/pN+LEAgfEyYTu5WQqy4u9/V/n6Rb+J81rcwru3 ED2BIQ9/WVkoxoOYIusUYNuI1ktydPsulTWHzr30WE8WyCWxhkHqDp2GT7f1oanzEMpn NAsCPPgXBk4xxTto3q5rwcFQ0aw7lKYt61hsi69GdErsJ6hTS0BI7OEluxRIOUyteJQ3 pU2oMY8p/LQmYZFR2fd05xSHouIBDUMJfV8FI2iFHfFJ+zvbzMSQEGvTCin7RmJps9QI YjfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=AgWYlRWMJwN7Q/fndV3qnR0W4Sg0AWhlDmt6bGcaOyQ=; b=KlMeYrPS8bYOxEa25l+ZQySCwQnlbuQiOp5Zzhx5RQF6X07jjnyniRQhFY7fJp3HnR y6sSgd5+kEIYuplOa4tQNCdwflsPmV7x+L0ELjNONQgG8ducxPN793jradGs6Z86wYxb HIsYGwYQuOUoK0fQr8LAZdu9JYJNipjzPV6ph2t+8RjePIJ1UHy5XhE82+0ARCPW27wR WTB2tuuMeQev8VZPjTkvLztdY5iA78xAVuJrcZqo+n+uEFoXZ8x8zH2/iMSQYje/Vm2F jigEc6YVdRfanmcNmzn7gvWpc11WKTriX+Ql38IjQoPS4qkzfVSt4Jn2bIWis1KNfQ1e BTzQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id m8si408443wmc.3.2019.05.22.21.44.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 May 2019 21:44:13 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id x4N4iC3G022963 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 23 May 2019 06:44:12 +0200 Received: from [167.87.2.45] ([167.87.2.45]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x4N4iCup008734 for ; Thu, 23 May 2019 06:44:12 +0200 Subject: Re: [DISCUSSIONS] CI build To: isar-users References: <20190522194410.GC2441@yssyq.m.ilbers.de> From: Jan Kiszka Message-ID: Date: Thu, 23 May 2019 06:44:12 +0200 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <20190522194410.GC2441@yssyq.m.ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: EFFyk5rcXh6o On 22.05.19 21:44, Baurzhan Ismagulov wrote: > Hi Claudius, > > thanks for summarizing. I think we'll open github issues at least for some of > the items. > > On Wed, May 22, 2019 at 09:26:51AM +0200, Claudius Heine wrote: >> Here are my issues of the current CI with Jenkins in text: >> - Unreliable build: >> -> Network and other unrelated issues to the patch >> -> Contains expected to fail tests > > We'll work on these. > > >> - Not transparent configuration/environment: >> -> parameters with which ci_build.sh is called are not controllable from >> the repository > > Hmm, the intention actually was that everyone runs the same fixed testsuites. > If desired, we could publish the arguments and gradually move them to the isar > repo. > > >> -> difficult to select which test case should be run > > This is the motivation for Avocado. > > >> -> Configuration of the environment is not controllable from the >> repository: Installed packages, System configuration (locales), >> Distribution, etc. > > Here the intention has also been to have one standard testing environment, and > it is Debian stable + instructions from Quick Start, nothing more. I'd welcome > moving to kas + docker. If desired, we could publish the current setup. > > I'd like to have the testsuite be runnable from any CI system with the same > script, without arguments. Then people could deploy any system they like > (Jenkins, GitLab, Travis CI, AWS...). The same testsuite should also be > runnable on one's host (already the case with ci_build, vm_smoke_test). Under > the hood, every test case should be defined and runnable individually > (Avocado). > > >> - Just one giant build process >> -> Takes a long time > > This will be solved with Avocado. > Nope, this requires scale-out into parallel jobs on a build farm, rather than a single server. E.g. we would scale better by doing per-arch builds so that arch commonality can still be exploited but the rest can work lock-free with own I/O bandwidth etc. on separate servers (or nodes in the cloud). > >> -> Cannot be spread over multiple servers > > Haven't looked at that, ideas welcome. Jan told us he is preparing something, > but load balancing would spread multiple pipelines, not tasks within one > pipeline. Depending on the costs, it might or might not be cheaper for us to > throw more physical servers on the task. > https://lists.cip-project.org/pipermail/cip-dev/2019-May/002371.html > >> -> Giant log file, that makes it difficult to figure out exactly which >> command or test case caused an issue to reproduce that locally > > This will be solved with Avocado. > > >> - Manually triggering of build jobs >> -> After pushing of CI branch, build has to be triggered via button > > I actually prefer working in this mode. Henning has polling enabled. If you > prefer two branches, please send the second branch URL to Maxim. > I also prefer standard build-on-push. But when everyone can build his own CI or use locally shared ones with personal, repo-controlled settings, we no longer depend on having to sync the working modes. > >> -> Patches on ML should be automatically tested and reported back to ML >> (that only makes sense if the build is reliable though) > > I haven't looked at that. Is there a tool that does this? > > >> - No "test everything" (acceptance test) build job > > Currently, it's the combination of cross and native builds. However, there are > a couple of steps that are still performed manually for various reasons. I'd > like to see them automated if at all possible. > > >> - Logs aren't public accessible > > Logs are accessible with authentication. Jenkins has public mode, but it lists > the users publicly, that is why we haven't opened it till now. If desired, we > could look at that. Is it that high-priority, given that all major contributors > here have access? > Let's stop wasting thoughts on legacy Jenkins, rather discuss how we can move on to gitlab-ci, if something is missing there etc. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux