From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6693749539433611264 X-Received: by 2002:a2e:3806:: with SMTP id f6mr9161469lja.33.1558518609913; Wed, 22 May 2019 02:50:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:6356:: with SMTP id x83ls185490ljb.7.gmail; Wed, 22 May 2019 02:50:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqxigsWT2/tNYCixAucrTB0bPs6JoX5Go58R7wW8Mlbn8gy5eneooQ8n7129oSjnHtPgEOvQ X-Received: by 2002:a2e:6545:: with SMTP id z66mr21159417ljb.146.1558518609315; Wed, 22 May 2019 02:50:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558518609; cv=none; d=google.com; s=arc-20160816; b=Vr7strI3672k8N/WzQil26ZEjAm6v3GGrOmel/RxU/+4PfmriOgfhGI8s92qBG+jSL m5/ePOk2+ZIhXKa34h+Sgc8valhL0XZycIwKH7785d6+0u02LrG0jQQmXgThgEmLJz9B It2TKBK+AwOkydyRda+oMCH//XexMQ2Y4z/SBGrHiLzBWbAi7SXJJyavqaj3nSUhNPUC +ETwkyIevFVz7PDDAv9GqPKHmRepPkBu/Q5AhG+wh0K1BkdDqIbsfgSZXpzA3u9u9nVc SmJKy6MKdGqkm2kkSCrvpWaG4/HIsw8g1XgTW9bauX6/o+c0nxX2zY3souI5Fd8x02fk 5eLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:to:subject; bh=du6qrP3ykhN1MSj+hwlJ/XXziAGqyTMh+O0nREdRX00=; b=mDdU4Rhflo1NJWi7awFLyoxfLVIxoJn7gxTth4/e5YVcg1lnUR3QBRXPNwrTPUtGCz qljZGOW9P/CbTwq3zAWv/Iz0/E8HHo+ffrsox6wfiqPQDIemSgJXAvwwnPdzof97oRhz Y+VDfd5b8shHvFMv41hSX5KildmsXvIlpiamBEyr5XNOgOpen3Vevsul/MdZMaPd8TpW dMS+FvddmkE4NJcz+p91SWt46chtO68yVkMcMva8+zPLSgIyGXZj5GUgnZbAFKWrrbH4 vO/9wIu6o1eVinAEivV0XmRZULc+vG6T6l7uaUtOdLnf5Cue+eX1seEl5NG99CbSzU7Y ncNA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id c9si2340457lff.2.2019.05.22.02.50.09 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 22 May 2019 02:50:09 -0700 (PDT) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from [192.168.1.29] (195.165-131-109.adsl-dyn.isp.belgacom.be [109.131.165.195] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x4M9o6Y7003978 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 22 May 2019 11:50:07 +0200 Subject: Re: [DISCUSSIONS] CI build To: Claudius Heine , isar-users References: <5128c754-55f8-924b-365c-2d529131bd3a@ilbers.de> <16e80d33-4f24-8b5a-de02-c02a90cc2c77@siemens.com> From: "Maxim Yu. Osipov" Organization: ilbers GmbH Message-ID: <57463c63-9747-126f-4b98-dc042f2b674a@ilbers.de> Date: Wed, 22 May 2019 11:50:00 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <16e80d33-4f24-8b5a-de02-c02a90cc2c77@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: vYh/rC6xJT7F On 5/22/19 11:10 AM, Claudius Heine wrote: > Hi Maxim, > > On 22/05/2019 10.20, Maxim Yu. Osipov wrote: >> On 5/22/19 9:26 AM, Claudius Heine wrote: > [...] >>>     -> Configuration of the environment is not controllable from the >>> repository: Installed packages, System configuration (locales), >>> Distribution, etc. >> >> In my opinion we need to have some "reference" common (better as >> minimal as possible) environment, to avoid a zoo of different >> environments... > > Well the more different environments to test against the more you can > find bugs. If we don't want that, then we should deliver a such and > environment together with isar. Please provide your detailed proposals how this can be formalized as there is an indefinite number of environments. Again, I vote for some common "reference" environment. > [...] > >> >>>     -> Cannot be spread over multiple servers >>>     -> Giant log file, that makes it difficult to figure out exactly >>> which command or test case caused an issue to reproduce that locally >> >> Your project is configured with -q (quiet) option and doesn't generate >> huge trace - it becomes verbose only on encountered errors - >> I had no problems with such log file as you can easily find the >> encountered error and always analyze the build workspace which is not >> cleaned up for the last build. >> >> Just have a look at your latest build >> http://isar-build.org:8080/job/isar_claudius_ilbers-ci/64/console >> it took for me 10 seconds to locate the failed recipe. > > Well its not always about a failed recipe. The ci_build.sh script now > contains multiple different calls to bitbake, but we don't see those > calls in the log. There is also no mention about other calls, like sed > which changes the configuration. OK, I'll add `set -x` in ci_build.sh. > >> >>>   - Manually triggering of build jobs >>>     -> After pushing of CI branch, build has to be triggered via button >> >> One may enable such build trigger (I believe that such trigger is >> enabled for Henning's project). but from my user's experience I prefer >> to start the build manually. > > I normally just push to the ci branch when I want to start tests. I've enabled the corresponding options for your projects - please try. > >> >> Of course, overnights builds are started periodically. >> >> See for details: >> https://wiki.jenkins.io/display/JENKINS/Building+a+software+project >> >>>     -> Patches on ML should be automatically tested and reported back >>> to ML (that only makes sense if the build is reliable though) >> >> Do we really want such feature? >> My concern is that ML will be polluted by such reports... > > You are currently testing and reporting back manually, if instead of you > some bot would do that automatically, then I don't how that would > increase the noise on the ML. But as I said that only makes sense if the > false positive and negative rates of CI is lessened. Manual work is unavoidable as sometimes patches are not merged properly. I'll enable reporting to ML when I push a patch set to maintainers branches for CI. Regards, Maxim. > > regards, > Claudius > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov