From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6693749539433611264 X-Received: by 2002:a2e:9a1a:: with SMTP id o26mr33561909lji.174.1558609119587; Thu, 23 May 2019 03:58:39 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:4212:: with SMTP id y18ls530842lfh.0.gmail; Thu, 23 May 2019 03:58:39 -0700 (PDT) X-Google-Smtp-Source: APXvYqy9qWHxSob390qOeLNza5Uo2tKQdnJVcjGbRs/tjAZ1HWIaZRLfdydbEGxf8NevF2s2MQu4 X-Received: by 2002:ac2:4c36:: with SMTP id u22mr1801248lfq.33.1558609119170; Thu, 23 May 2019 03:58:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558609119; cv=none; d=google.com; s=arc-20160816; b=gbHRPKdGg6jjwL86wbboQBIuI9v9HZsUtgxo+xDRwPIP2+bTk/gxui/+uwxnafRbJP Rm4VYltIwJUQPBC87zdsFz4Stpzmlq5Cbq+5VDRueWZcsN+WHjQ0BGl9NbRqjoyfEDb0 hI5zhTxA6jL2Ew+NtNLkP6mBf3Cmjn80JxLgH+WIxhjcoaZvH8Vu57foC0X++6a4lvk6 QW4REPZcu9yvqNOmJoYZuT6A1D9Qewd2b+uA9GCir4746drzQTyGCyKahZh5yWVo/BMs rri/mT63D03x6MIEK2qSmoUK48VxhjYxTpQTMUJmvFOXuXnw21R+/xRW+Wav+yElQYLo r9Uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:to:subject; bh=X8Wm3svcpAEYWaPJQMnMQLBuGJPYPn7oQJ9zvjZfz7U=; b=tcWrN5ZZQlVvBWd2QZvNZ01RPTJZCDgoOxBFaoi7T5cgBolDWzXK20TFjibKqN4CCL zVJoF/FGyzSu/JrIHm9nHiZPjFoTyxhMlpiuS3Y8atU+JyhZzax8WV/jb9aa15Ki1gSO ko18LRwkcPbAOVNE1+W2iiUmq0IknRkEFm6lzGIplSCvgdbk042grxPJ6whhexlw2M7g QKbcbJtIC8j+VGFzpW78jmbOQ6sWPr+6oQFI99hanpGUazcIS5uUl94ApJewYWKzi7wx BswxHaSbZt0AVbBCaA49S3n3ewVwM22yj0KnQPFeewIVEGY1lZtHDV5kYdnwPVbcDLuM 6SSQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id o20si2066890lji.2.2019.05.23.03.58.38 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 03:58:39 -0700 (PDT) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from [192.168.1.29] (195.165-131-109.adsl-dyn.isp.belgacom.be [109.131.165.195] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x4NAwaDo008877 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 May 2019 12:58:36 +0200 Subject: Re: [DISCUSSIONS] CI build To: Claudius Heine , "[ext] Jan Kiszka" , isar-users References: <20190522194410.GC2441@yssyq.m.ilbers.de> From: "Maxim Yu. Osipov" Organization: ilbers GmbH Message-ID: <45df5492-52bc-5e47-04f5-9700920a66e7@ilbers.de> Date: Thu, 23 May 2019 12:58:30 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 8bRMofPs7545 On 5/23/19 10:32 AM, Claudius Heine wrote: > Hi Jan, hi Baurzhan, > > On 23/05/2019 06.44, [ext] Jan Kiszka wrote: >>>>   - Not transparent configuration/environment: >>>>     -> parameters with which ci_build.sh is called are not >>>> controllable from >>>> the repository >>> >>> Hmm, the intention actually was that everyone runs the same fixed >>> testsuites. >>> If desired, we could publish the arguments and gradually move them to >>> the isar >>> repo. > > Which is currently not the case, we have a 'normal' build, a 'fast' > build, a 'cross' build and a 'repo' build, which can be combined with > 'normal', 'fast' and 'cross' IIUC. Just to clarify. If the patches I've sent yesterday are accepted the 'normal' build will include * parallel cross compilation of the set stretch qemuarm/qemuarm64/qemuamd64/de0-nano-soc/rpi and buster qemuarm * sdk creation for stretch qemuarm * parallel native compilation of the set stretch qemuarm/qemuarm64/qemui386/qemuamd64/rpi and buster qemuarm/qemui386/qemuamd64/qemuamd64-buster-tgz/nand-ubi-demo 'fast' build excludes native compilation. If the extra option 'repro' is enabled, additional separate build to create/use cached base repo for stretch qemuarm/qemuarm64/qemuamd64 is launched. To maximize test coverage I've enabled repro for all your projects in Jenkins. So if the patch set is accepted it would be enough to run 'normal' build with 'repro' option: 'scripts/ci_build.sh -q -r' Regards, Maxim. >>> >>> >>>>     -> difficult to select which test case should be run >>> >>> This is the motivation for Avocado. >>> >>> >>>>     -> Configuration of the environment is not controllable from the >>>> repository: Installed packages, System configuration (locales), >>>> Distribution, etc. >>> >>> Here the intention has also been to have one standard testing >>> environment, and >>> it is Debian stable + instructions from Quick Start, nothing more. >>> I'd welcome >>> moving to kas + docker. If desired, we could publish the current setup. >>> >>> I'd like to have the testsuite be runnable from any CI system with >>> the same >>> script, without arguments. Then people could deploy any system they like >>> (Jenkins, GitLab, Travis CI, AWS...). The same testsuite should also be >>> runnable on one's host (already the case with ci_build, >>> vm_smoke_test). Under >>> the hood, every test case should be defined and runnable individually >>> (Avocado). > > Personally I don't really care about executing the whole CI process > locally. I am much more interested in triggering single failed > test-cases locally to work on them. So the 'ci_build.sh' is not really > useful for me. > > 'ci_build.sh' is just to unhandy to use locally. > > [...] >>>>     -> Cannot be spread over multiple servers >>> >>> Haven't looked at that, ideas welcome. Jan told us he is preparing >>> something, >>> but load balancing would spread multiple pipelines, not tasks within one >>> pipeline. Depending on the costs, it might or might not be cheaper >>> for us to >>> throw more physical servers on the task. >>> >> >> https://lists.cip-project.org/pipermail/cip-dev/2019-May/002371.html > > Would have been nice to have this cross-posted to this ML as well. > > Will Michael write patches for isar to integrate this, or does he needs > anything to do this? > >>> >>>>     -> Patches on ML should be automatically tested and reported >>>> back to ML >>>> (that only makes sense if the build is reliable though) >>> >>> I haven't looked at that. Is there a tool that does this? > > Not really. But I posted a patchwork+jenkins integration howto some time > ago: https://that.guru/blog/patchwork-and-ci-in-a-tree/ > >>>>   - Logs aren't public accessible >>> >>> Logs are accessible with authentication. Jenkins has public mode, but >>> it lists >>> the users publicly, that is why we haven't opened it till now. If >>> desired, we >>> could look at that. Is it that high-priority, given that all major >>> contributors >>> here have access? >>> >> >> Let's stop wasting thoughts on legacy Jenkins, rather discuss how we >> can move on to gitlab-ci, if something is missing there etc. > > I can only figure out what is missing after I tried it out. > > regards, > Claudius > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov