From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6680483261914808320 X-Received: by 2002:adf:db0b:: with SMTP id s11mr3441654wri.180.1556184827354; Thu, 25 Apr 2019 02:33:47 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:4d51:: with SMTP id a17ls210304wru.4.gmail; Thu, 25 Apr 2019 02:33:46 -0700 (PDT) X-Google-Smtp-Source: APXvYqxUjZXMMJY0FSO+rrY17ti7z1j9/QBhe38rceJ/xL1+KvylBFCV5MBHodfsNPH7LXUf3L/9 X-Received: by 2002:adf:cf0c:: with SMTP id o12mr25188948wrj.16.1556184826728; Thu, 25 Apr 2019 02:33:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1556184826; cv=none; d=google.com; s=arc-20160816; b=sipPqdvzcMIS6v55s6SYj0njLUIrM/mXdmNqj9mBYOxbcPmDIJ3rbKRj3/JGnhnx+S NDYYyoelYgQKsUqAXHQ+0nyyyjbXc3LYrwoYSgDYMOm5JHeqox/naa8tmzTGbCFXrLd5 PB8wyk4No2hGM2VmtRIZQevofUOx1IsuqELA6Cme/Gy4cigFK1v2JNto49kF/3yj98RG FIXCcd/iJKhvitErf3drDtLIHwKZsCLuUUmKNDjdvl0I2CeoyBQhgY/44D4363MqmP3A U/yohS25i3nK0IEWxF/32QHxysIS/GdbE66hi+a+wjg83gO6E14oE6atHAqNl5gD9ZE9 aHRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:cc:to :subject; bh=d7S7nRIltBtj38eiE4CVROqglreGQGmM1ZBAeVtnyTY=; b=HlJ0G6M962YK4WNwOcrAElmyunFF6+JVcSqcTalplzUSmkpf3zVx8SG06aEV2BaDMo 1L/bfHLCi8uL29+rTlYgEFGfbykhVJ8Sb0CePMqHeXyyf+nDwHcANGfE6S5Hwy1+DrDX BCWgYY6NVk5+HD9M680zFdV5trZcVVwHOe20zoO+2R7ljktutXdV80ZbDBtIVlLcmYhd HvXjYpWj48OrCJwpnVcAcaqn+dPRL2AUrV4SJVGov3O0Pj8dIHJDHJKBQAPm/XCQx8Ip LzTu7TNJGBJVmtegy6+mIK/jkdhEmCo/bUIneIKQjR7/L7JY01iySgoHi9+uC/aNcWZ5 ZdQw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id t17si1190120wri.5.2019.04.25.02.33.46 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 Apr 2019 02:33:46 -0700 (PDT) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from [192.168.50.180] (nat-ppp-217.71.235.199-satnet-spb.ru [217.71.235.199] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x3P9Xef5009090 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 25 Apr 2019 11:33:42 +0200 Subject: Re: [PATCH v3 0/8] Cleanup rootfs creation To: Claudius Heine , isar-users@googlegroups.com Cc: Claudius Heine References: <20190424092734.14167-1-claudius.heine.ext@siemens.com> <33359f72-d7d0-0d41-051d-da9885bd7354@ilbers.de> <0523bc46-f96e-d4d0-1828-fc99079c938b@siemens.com> <561243bd-3aa7-5f8a-83c5-4137660354fb@siemens.com> <1aa42268-5f7e-355b-e980-eebe9df4a526@ilbers.de> <3a1376a4-6a66-c71a-9565-b4e77b535509@ilbers.de> <6f41982e-7b01-d02c-7785-297061eb8cd4@siemens.com> From: "Maxim Yu. Osipov" Organization: ilbers GmbH Message-ID: <7ff79c17-aa00-f18f-a6ad-4d0f16492eac@ilbers.de> Date: Thu, 25 Apr 2019 12:33:34 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <6f41982e-7b01-d02c-7785-297061eb8cd4@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: RaMSeXLaWHxi On 4/25/19 11:11 AM, Claudius Heine wrote: > On 25/04/2019 10.39, Maxim Yu. Osipov wrote: >> On 4/25/19 10:27 AM, Claudius Heine wrote: >>> On 25/04/2019 10.07, Maxim Yu. Osipov wrote: >>>> Hi Claudius, >>>> >>>> On 4/25/19 10:01 AM, Claudius Heine wrote: >>>>> Hi Maxim, >>>>> >>>>> On 25/04/2019 09.55, Maxim Yu. Osipov wrote: >>>>>> Hi Claudius, >>>>>> >>>>>> On 4/25/19 8:40 AM, Claudius Heine wrote: >>>>>>> Hi Maxim, >>>>>>> >>>>>>> On 25/04/2019 08.03, Maxim Yu. Osipov wrote: >>>>>>>> >>>>>>>> Tried this series with fast ci build ("scripts/ci_build.sh -q -f"- >>>>>>>> similar problems as before. >>>>>>>> http://isar-build.org:8080/job/isar_mosipov_develop/98/console >>>>>>>> >>>>>>>> I observe the same errors on my host. >>>>>>> >>>>>>> So what do we do then when the CI is not reproducible? >>>>>>> >>>>>>> I am at a loss at how to investigate an error that does not >>>>>>> happen when I build on isar-build or on my local host and only >>>>>>> when you build it on isar-build or at your host. >>>>>> >>>>>> Do you mean that this problem is not reproduced when you run >>>>>> "fast" CI build (ci_build.sh -q -f) on your PC? >>>>> >>>>> I don't do ci_builds locally, because that is the job of the CI. I >>>>> just build one target that might be affected of the specific change. >>>> >>>> That's the reason why you can't reproduce the problem. >>> >>> Well I did build the multiconfig+target where the issue occured IIRC. > > I looked into the "fast" build now, and it cross-compiles > "multiconfig:rpi-stretch:isar-image-base". Now I tried that and the > build fails even on the current 'next' so the issue seems to be there. No. "Fast" CI build (which in particular tests cross-compilation) in 'next' is OK - I've built it today morning when I tested your patch "ubi-img/ubifs-img: change fatal error to skip recipe" and my patch which includes SDK cretion test into "fast" build. You may look at the log: http://isar-build.org:8080/job/isar_mosipov_develop/97/console >>> >>>> >>>>>> >>>>>>> Any suggestions? >>>>>> >>>>>> Yes, sure. We will create an additional project for you on >>>>>> isar-build.org to launch "fast" CI build. >>>>> >>>>> So you suggest that I should now test every patch I send two times, >>>>> once for each different CI configuration manually? >>>> >>>> There is another option. I may reconfigure your current project to >>>> fast build. Should I do that? >>> >>> I don't care. I would like to have my CI configured in a way that >>> allows my patches to tested for upstream inclusion. If that is in a >>> "fast", "slow" or "totally unnoteworthy" velocity does not matter for >>> that. CI is an acceptability test and I want the strictest one. If I >>> was mistaken that "fast" is less strict than "slow", then switch to >>> "fast". If I now need to pass "fast" and "slow" for my patch to be >>> acceptable, then I would suggest introducing a "abysmal slow" CI run >>> that combines "slow" and "fast". >> >> I've just sent the CI tests procedure in separate email. > > That is not a CI test procedure. That is a test procedure for local > testing, if someone wants to use the ci scripts to do so. I don't since > my environment is different from what you have on your CI server. > >> The problem that if we include all the features into single CI run it >> will takes hours to test single patch. > > Right. "abysmal slow". But in your 'CONTRIBUTERS' you state that > everyone should do that anyway. Manually. It's highly suggested but its' not a must. Current iteration of patch series just proves that it's worth to run these scripts if you want to avoid multiple patches versions iterations. > >> >> Anyway I pointed you to the problem and the way how to reproduce it. >> You wrote that you don't want to call CI locally. It's up to you - I >> don't care too - if the problem is not fixed the patchset will not be >> applied. > > Well I don't have the same environment as you on your CI server, so > running scripts build for your CI server is not something I can do. That > is not choice that is just how things are. > >> I didn't understood from your emotional passage what configuration you >> prefer. > > Well CI testing should be done reproducible and transparent. If my > branch on jenkins has a different configuration than some other branch > there and now the invisible configuration of those branches changes like > the weather, then this is not reproducible nor transparent. > > Having to explain the basics of what CI is, is pretty nerve wreaking and > exhausting. Thank you for explaining what CI is - I'm sorry to make you nervous and exhausted. >> I  suggested two options >> >> 1) Add additional project for you for "fast" build. Yes. In such cases >> you have to launch the tests in both projects but you will be sure that >> integration will go more smoothly. > > Well the most acceptable way would be 3 branches. One for "fast", one > for "slow" and one for "acceptance" (which combines slow and fast). > >> >> 2) Reconfigure your existing project from "standard" to the "fast" one. >> >> It is up to you to decide how you want to test your patch set. > > No its not. It is up to you to test the patchset and decide if that > should be merged or not. So contributors have to jump through those > hoops in order for it to be merged reasonable fast. And I decided to don't merge your patch set because tests were failed. That's the only story. Maxim. > Claudius > >> >> Greetings, >> Maxim. >> >> >>> Claudius >>> >>>> >>>> Maxim. >>>> >>>>> Claudius >>>>> >>>>>> >>>>>> Maxim. >>>>>> >>>>>>> Claudius >>>>>>> >>>>>>>> >>>>>>>> Maxim. >>>>>>>> >>>>>>>> On 4/24/19 11:27 AM, claudius.heine.ext@siemens.com wrote: >>>>>>>>> From: Claudius Heine >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I could not reproduce the raspberrypi issue Maxim found in his >>>>>>>>> CI build: >>>>>>>>> http://isar-build.org:8080/job/isar_mosipov_develop/91/consoleText >>>>>>>>> >>>>>>>>> This is my build log: >>>>>>>>> http://isar-build.org:8080/job/isar_claudius_ilbers-ci/54/consoleText >>>>>>>>> >>>>>>>>> >>>>>>>>> Only the module loading fails, which Maxim stated he will look >>>>>>>>> into. >>>>>>>>> >>>>>>>>> This patchset uses still the 'old' style for >>>>>>>>> refactoring/cleanup patchsets, >>>>>>>>> until I automated the official patch submission process as >>>>>>>>> documented by the >>>>>>>>> CONTRIBUTERS policy. >>>>>>>>> >>>>>>>>> You can find the whole patchset in under this url: >>>>>>>>> >>>>>>>>> https://github.com/cmhe/isar/tree/ch/cleanup >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Claudius >>>>>>>>> >>>>>>>>> changes from v2: >>>>>>>>>   - replaced 'ROOTFS_ARCH' with 'HOST_ARCH' in sdkchroot.bb to >>>>>>>>> avoid >>>>>>>>>     populate_sdk issue >>>>>>>>> >>>>>>>>> changes from v1: >>>>>>>>>   - fixed typo in commit message >>>>>>>>> >>>>>>>>> Claudius Heine (8): >>>>>>>>>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to >>>>>>>>>      base.bbclass >>>>>>>>>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to >>>>>>>>> isar-bootstrap >>>>>>>>>    buildchroot.bbclass: only cross build if HOST_ARCH != >>>>>>>>> DISTRO_ARCH >>>>>>>>>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR >>>>>>>>>    bitbake.conf: remove unneeded and differently used variables >>>>>>>>>    image.bbclass: make IMAGE_ROOTFS overwritable >>>>>>>>>    bitbake.conf: set default QEMU_ARCH variables >>>>>>>>>    buildchroot/configscript: make creation of builder uid/gid >>>>>>>>> idempotent >>>>>>>>> >>>>>>>>>   meta/classes/base.bbclass                          |  9 >>>>>>>>> +++++++++ >>>>>>>>>   meta/classes/buildchroot.bbclass                   |  2 +- >>>>>>>>>   meta/classes/image-sdk-extension.bbclass           |  2 +- >>>>>>>>>   meta/classes/image.bbclass                         |  2 +- >>>>>>>>>   meta/classes/isar-bootstrap-helper.bbclass         | 14 >>>>>>>>> -------------- >>>>>>>>>   meta/conf/bitbake.conf                             | 13 >>>>>>>>> ++++++++----- >>>>>>>>>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 >>>>>>>>> +++-------- >>>>>>>>>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +----- >>>>>>>>>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 + >>>>>>>>>   .../buildchroot/buildchroot-host.bb                |  1 + >>>>>>>>>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +-- >>>>>>>>>   .../buildchroot/files/configscript.sh              |  4 ++-- >>>>>>>>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +-- >>>>>>>>>   13 files changed, 30 insertions(+), 41 deletions(-) >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov