Re: [DISCUSSIONS] CI build

[Jan Kiszka <jan.kiszka@siemens.com>]

On 22.05.19 21:44, Baurzhan Ismagulov wrote:
> Hi Claudius,
> 
> thanks for summarizing. I think we'll open github issues at least for some of
> the items.
> 
> On Wed, May 22, 2019 at 09:26:51AM +0200, Claudius Heine wrote:
>> Here are my issues of the current CI with Jenkins in text:
>>   - Unreliable build:
>>     -> Network and other unrelated issues to the patch
>>     -> Contains expected to fail tests
> 
> We'll work on these.
> 
> 
>>   - Not transparent configuration/environment:
>>     -> parameters with which ci_build.sh is called are not controllable from
>> the repository
> 
> Hmm, the intention actually was that everyone runs the same fixed testsuites.
> If desired, we could publish the arguments and gradually move them to the isar
> repo.
> 
> 
>>     -> difficult to select which test case should be run
> 
> This is the motivation for Avocado.
> 
> 
>>     -> Configuration of the environment is not controllable from the
>> repository: Installed packages, System configuration (locales),
>> Distribution, etc.
> 
> Here the intention has also been to have one standard testing environment, and
> it is Debian stable + instructions from Quick Start, nothing more. I'd welcome
> moving to kas + docker. If desired, we could publish the current setup.
> 
> I'd like to have the testsuite be runnable from any CI system with the same
> script, without arguments. Then people could deploy any system they like
> (Jenkins, GitLab, Travis CI, AWS...). The same testsuite should also be
> runnable on one's host (already the case with ci_build, vm_smoke_test). Under
> the hood, every test case should be defined and runnable individually
> (Avocado).
> 
> 
>>   - Just one giant build process
>>     -> Takes a long time
> 
> This will be solved with Avocado.
> 

Nope, this requires scale-out into parallel jobs on a build farm, rather than a 
single server. E.g. we would scale better by doing per-arch builds so that arch 
commonality can still be exploited but the rest can work lock-free with own I/O 
bandwidth etc. on separate servers (or nodes in the cloud).

> 
>>     -> Cannot be spread over multiple servers
> 
> Haven't looked at that, ideas welcome. Jan told us he is preparing something,
> but load balancing would spread multiple pipelines, not tasks within one
> pipeline. Depending on the costs, it might or might not be cheaper for us to
> throw more physical servers on the task.
> 

https://lists.cip-project.org/pipermail/cip-dev/2019-May/002371.html

> 
>>     -> Giant log file, that makes it difficult to figure out exactly which
>> command or test case caused an issue to reproduce that locally
> 
> This will be solved with Avocado.
> 
> 
>>   - Manually triggering of build jobs
>>     -> After pushing of CI branch, build has to be triggered via button
> 
> I actually prefer working in this mode. Henning has polling enabled. If you
> prefer two branches, please send the second branch URL to Maxim.
> 

I also prefer standard build-on-push. But when everyone can build his own CI or 
use locally shared ones with personal, repo-controlled settings, we no longer 
depend on having to sync the working modes.

> 
>>     -> Patches on ML should be automatically tested and reported back to ML
>> (that only makes sense if the build is reliable though)
> 
> I haven't looked at that. Is there a tool that does this?
> 
> 
>>   - No "test everything" (acceptance test) build job
> 
> Currently, it's the combination of cross and native builds. However, there are
> a couple of steps that are still performed manually for various reasons. I'd
> like to see them automated if at all possible.
> 
> 
>>   - Logs aren't public accessible
> 
> Logs are accessible with authentication. Jenkins has public mode, but it lists
> the users publicly, that is why we haven't opened it till now. If desired, we
> could look at that. Is it that high-priority, given that all major contributors
> here have access?
> 

Let's stop wasting thoughts on legacy Jenkins, rather discuss how we can move on 
to gitlab-ci, if something is missing there etc.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux