Re: [PATCH v3 0/8] Cleanup rootfs creation

["Maxim Yu. Osipov" <mosipov@ilbers.de>]

On 4/25/19 11:11 AM, Claudius Heine wrote:
> On 25/04/2019 10.39, Maxim Yu. Osipov wrote:
>> On 4/25/19 10:27 AM, Claudius Heine wrote:
>>> On 25/04/2019 10.07, Maxim Yu. Osipov wrote:
>>>> Hi Claudius,
>>>>
>>>> On 4/25/19 10:01 AM, Claudius Heine wrote:
>>>>> Hi Maxim,
>>>>>
>>>>> On 25/04/2019 09.55, Maxim Yu. Osipov wrote:
>>>>>> Hi Claudius,
>>>>>>
>>>>>> On 4/25/19 8:40 AM, Claudius Heine wrote:
>>>>>>> Hi Maxim,
>>>>>>>
>>>>>>> On 25/04/2019 08.03, Maxim Yu. Osipov wrote:
>>>>>>>>
>>>>>>>> Tried this series with fast ci build ("scripts/ci_build.sh -q -f"-
>>>>>>>> similar problems as before.
>>>>>>>> http://isar-build.org:8080/job/isar_mosipov_develop/98/console
>>>>>>>>
>>>>>>>> I observe the same errors on my host.
>>>>>>>
>>>>>>> So what do we do then when the CI is not reproducible?
>>>>>>>
>>>>>>> I am at a loss at how to investigate an error that does not 
>>>>>>> happen when I build on isar-build or on my local host and only 
>>>>>>> when you build it on isar-build or at your host.
>>>>>>
>>>>>> Do you mean that this problem is not reproduced when you run 
>>>>>> "fast" CI build (ci_build.sh -q -f) on your PC?
>>>>>
>>>>> I don't do ci_builds locally, because that is the job of the CI. I 
>>>>> just build one target that might be affected of the specific change.
>>>>
>>>> That's the reason why you can't reproduce the problem.
>>>
>>> Well I did build the multiconfig+target where the issue occured IIRC.
> 
> I looked into the "fast" build now, and it cross-compiles 
> "multiconfig:rpi-stretch:isar-image-base". Now I tried that and the 
> build fails even on the current 'next' so the issue seems to be there.

No. "Fast" CI build (which in particular tests cross-compilation) in
'next' is OK - I've built it today morning when I tested your patch
"ubi-img/ubifs-img: change fatal error to skip recipe" and my patch 
which includes SDK cretion test into "fast" build.

You may look at the log:
http://isar-build.org:8080/job/isar_mosipov_develop/97/console

>>>
>>>>
>>>>>>
>>>>>>> Any suggestions?
>>>>>>
>>>>>> Yes, sure. We will create an additional project for you on 
>>>>>> isar-build.org to launch "fast" CI build.
>>>>>
>>>>> So you suggest that I should now test every patch I send two times, 
>>>>> once for each different CI configuration manually?
>>>>
>>>> There is another option. I may reconfigure your current project to 
>>>> fast build. Should I do that?
>>>
>>> I don't care. I would like to have my CI configured in a way that 
>>> allows my patches to tested for upstream inclusion. If that is in a 
>>> "fast", "slow" or "totally unnoteworthy" velocity does not matter for 
>>> that. CI is an acceptability test and I want the strictest one. If I 
>>> was mistaken that "fast" is less strict than "slow", then switch to 
>>> "fast". If I now need to pass "fast" and "slow" for my patch to be 
>>> acceptable, then I would suggest introducing a "abysmal slow" CI run 
>>> that combines "slow" and "fast".
>>
>> I've just sent the CI tests procedure in separate email.
> 
> That is not a CI test procedure. That is a test procedure for local 
> testing, if someone wants to use the ci scripts to do so. I don't since 
> my environment is different from what you have on your CI server.
> 
>> The problem that if we include all the features into single CI run it 
>> will takes hours to test single patch.
> 
> Right. "abysmal slow". But in your 'CONTRIBUTERS' you state that 
> everyone should do that anyway. Manually.

It's highly suggested but its' not a must.
Current iteration of patch series just proves that it's worth to run 
these scripts if you want to avoid multiple patches versions iterations.
> 
>>
>> Anyway I pointed you to the problem and the way how to reproduce it.
>> You wrote that you don't want to call CI locally. It's up to you - I 
>> don't care too - if the problem is not fixed the patchset will not be 
>> applied.
> 
> Well I don't have the same environment as you on your CI server, so 
> running scripts build for your CI server is not something I can do. That 
> is not choice that is just how things are.
> 
>> I didn't understood from your emotional passage what configuration you 
>> prefer.
> 
> Well CI testing should be done reproducible and transparent. If my 
> branch on jenkins has a different configuration than some other branch 
> there and now the invisible configuration of those branches changes like 
> the weather, then this is not reproducible nor transparent.
> 
> Having to explain the basics of what CI is, is pretty nerve wreaking and 
> exhausting.

Thank you for explaining what CI is - I'm sorry to make you nervous and 
exhausted.

>> I  suggested two options
>>
>> 1) Add additional project for you for "fast" build. Yes. In such cases 
>> you have to launch the tests in both projects but you will be sure that
>> integration will go more smoothly.
> 
> Well the most acceptable way would be 3 branches. One for "fast", one 
> for "slow" and one for "acceptance" (which combines slow and fast).
> 
>>
>> 2) Reconfigure your existing project from "standard" to the "fast" one.
>>
>> It is up to you to decide how you want to test your patch set.
> 
> No its not. It is up to you to test the patchset and decide if that 
> should be merged or not. So contributors have to jump through those 
> hoops in order for it to be merged reasonable fast.

And I decided to don't merge your patch set because tests were failed.
That's the only story.

Maxim.


> Claudius
> 
>>
>> Greetings,
>> Maxim.
>>
>>
>>> Claudius
>>>
>>>>
>>>> Maxim.
>>>>
>>>>> Claudius
>>>>>
>>>>>>
>>>>>> Maxim.
>>>>>>
>>>>>>> Claudius
>>>>>>>
>>>>>>>>
>>>>>>>> Maxim.
>>>>>>>>
>>>>>>>> On 4/24/19 11:27 AM, claudius.heine.ext@siemens.com wrote:
>>>>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I could not reproduce the raspberrypi issue Maxim found in his 
>>>>>>>>> CI build:
>>>>>>>>> http://isar-build.org:8080/job/isar_mosipov_develop/91/consoleText
>>>>>>>>>
>>>>>>>>> This is my build log:
>>>>>>>>> http://isar-build.org:8080/job/isar_claudius_ilbers-ci/54/consoleText 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Only the module loading fails, which Maxim stated he will look 
>>>>>>>>> into.
>>>>>>>>>
>>>>>>>>> This patchset uses still the 'old' style for 
>>>>>>>>> refactoring/cleanup patchsets,
>>>>>>>>> until I automated the official patch submission process as 
>>>>>>>>> documented by the
>>>>>>>>> CONTRIBUTERS policy.
>>>>>>>>>
>>>>>>>>> You can find the whole patchset in under this url:
>>>>>>>>>
>>>>>>>>> https://github.com/cmhe/isar/tree/ch/cleanup
>>>>>>>>>
>>>>>>>>> regards,
>>>>>>>>> Claudius
>>>>>>>>>
>>>>>>>>> changes from v2:
>>>>>>>>>   - replaced 'ROOTFS_ARCH' with 'HOST_ARCH' in sdkchroot.bb to 
>>>>>>>>> avoid
>>>>>>>>>     populate_sdk issue
>>>>>>>>>
>>>>>>>>> changes from v1:
>>>>>>>>>   - fixed typo in commit message
>>>>>>>>>
>>>>>>>>> Claudius Heine (8):
>>>>>>>>>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>>>>>>>>>      base.bbclass
>>>>>>>>>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to 
>>>>>>>>> isar-bootstrap
>>>>>>>>>    buildchroot.bbclass: only cross build if HOST_ARCH != 
>>>>>>>>> DISTRO_ARCH
>>>>>>>>>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>>>>>>>>>    bitbake.conf: remove unneeded and differently used variables
>>>>>>>>>    image.bbclass: make IMAGE_ROOTFS overwritable
>>>>>>>>>    bitbake.conf: set default QEMU_ARCH variables
>>>>>>>>>    buildchroot/configscript: make creation of builder uid/gid 
>>>>>>>>> idempotent
>>>>>>>>>
>>>>>>>>>   meta/classes/base.bbclass                          |  9 
>>>>>>>>> +++++++++
>>>>>>>>>   meta/classes/buildchroot.bbclass                   |  2 +-
>>>>>>>>>   meta/classes/image-sdk-extension.bbclass           |  2 +-
>>>>>>>>>   meta/classes/image.bbclass                         |  2 +-
>>>>>>>>>   meta/classes/isar-bootstrap-helper.bbclass         | 14 
>>>>>>>>> --------------
>>>>>>>>>   meta/conf/bitbake.conf                             | 13 
>>>>>>>>> ++++++++-----
>>>>>>>>>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 
>>>>>>>>> +++--------
>>>>>>>>>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
>>>>>>>>>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
>>>>>>>>>   .../buildchroot/buildchroot-host.bb                |  1 +
>>>>>>>>>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
>>>>>>>>>   .../buildchroot/files/configscript.sh              |  4 ++--
>>>>>>>>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
>>>>>>>>>   13 files changed, 30 insertions(+), 41 deletions(-)
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov