Re: [PATCH v3 0/8] Cleanup rootfs creation

[Claudius Heine <claudius.heine.ext@siemens.com>]

On 25/04/2019 10.39, Maxim Yu. Osipov wrote:
> On 4/25/19 10:27 AM, Claudius Heine wrote:
>> On 25/04/2019 10.07, Maxim Yu. Osipov wrote:
>>> Hi Claudius,
>>>
>>> On 4/25/19 10:01 AM, Claudius Heine wrote:
>>>> Hi Maxim,
>>>>
>>>> On 25/04/2019 09.55, Maxim Yu. Osipov wrote:
>>>>> Hi Claudius,
>>>>>
>>>>> On 4/25/19 8:40 AM, Claudius Heine wrote:
>>>>>> Hi Maxim,
>>>>>>
>>>>>> On 25/04/2019 08.03, Maxim Yu. Osipov wrote:
>>>>>>>
>>>>>>> Tried this series with fast ci build ("scripts/ci_build.sh -q -f"-
>>>>>>> similar problems as before.
>>>>>>> http://isar-build.org:8080/job/isar_mosipov_develop/98/console
>>>>>>>
>>>>>>> I observe the same errors on my host.
>>>>>>
>>>>>> So what do we do then when the CI is not reproducible?
>>>>>>
>>>>>> I am at a loss at how to investigate an error that does not happen 
>>>>>> when I build on isar-build or on my local host and only when you 
>>>>>> build it on isar-build or at your host.
>>>>>
>>>>> Do you mean that this problem is not reproduced when you run "fast" 
>>>>> CI build (ci_build.sh -q -f) on your PC?
>>>>
>>>> I don't do ci_builds locally, because that is the job of the CI. I 
>>>> just build one target that might be affected of the specific change.
>>>
>>> That's the reason why you can't reproduce the problem.
>>
>> Well I did build the multiconfig+target where the issue occured IIRC.

I looked into the "fast" build now, and it cross-compiles 
"multiconfig:rpi-stretch:isar-image-base". Now I tried that and the 
build fails even on the current 'next' so the issue seems to be there.

>>
>>>
>>>>>
>>>>>> Any suggestions?
>>>>>
>>>>> Yes, sure. We will create an additional project for you on 
>>>>> isar-build.org to launch "fast" CI build.
>>>>
>>>> So you suggest that I should now test every patch I send two times, 
>>>> once for each different CI configuration manually?
>>>
>>> There is another option. I may reconfigure your current project to 
>>> fast build. Should I do that?
>>
>> I don't care. I would like to have my CI configured in a way that 
>> allows my patches to tested for upstream inclusion. If that is in a 
>> "fast", "slow" or "totally unnoteworthy" velocity does not matter for 
>> that. CI is an acceptability test and I want the strictest one. If I 
>> was mistaken that "fast" is less strict than "slow", then switch to 
>> "fast". If I now need to pass "fast" and "slow" for my patch to be 
>> acceptable, then I would suggest introducing a "abysmal slow" CI run 
>> that combines "slow" and "fast".
> 
> I've just sent the CI tests procedure in separate email.

That is not a CI test procedure. That is a test procedure for local 
testing, if someone wants to use the ci scripts to do so. I don't since 
my environment is different from what you have on your CI server.

> The problem that if we include all the features into single CI run it 
> will takes hours to test single patch.

Right. "abysmal slow". But in your 'CONTRIBUTERS' you state that 
everyone should do that anyway. Manually.

> 
> Anyway I pointed you to the problem and the way how to reproduce it.
> You wrote that you don't want to call CI locally. It's up to you - I 
> don't care too - if the problem is not fixed the patchset will not be 
> applied.

Well I don't have the same environment as you on your CI server, so 
running scripts build for your CI server is not something I can do. That 
is not choice that is just how things are.

> I didn't understood from your emotional passage what configuration you 
> prefer.

Well CI testing should be done reproducible and transparent. If my 
branch on jenkins has a different configuration than some other branch 
there and now the invisible configuration of those branches changes like 
the weather, then this is not reproducible nor transparent.

Having to explain the basics of what CI is, is pretty nerve wreaking and 
exhausting.

> I  suggested two options
> 
> 1) Add additional project for you for "fast" build. Yes. In such cases 
> you have to launch the tests in both projects but you will be sure that
> integration will go more smoothly.

Well the most acceptable way would be 3 branches. One for "fast", one 
for "slow" and one for "acceptance" (which combines slow and fast).

> 
> 2) Reconfigure your existing project from "standard" to the "fast" one.
> 
> It is up to you to decide how you want to test your patch set.

No its not. It is up to you to test the patchset and decide if that 
should be merged or not. So contributors have to jump through those 
hoops in order for it to be merged reasonable fast.

Claudius

> 
> Greetings,
> Maxim.
> 
> 
>> Claudius
>>
>>>
>>> Maxim.
>>>
>>>> Claudius
>>>>
>>>>>
>>>>> Maxim.
>>>>>
>>>>>> Claudius
>>>>>>
>>>>>>>
>>>>>>> Maxim.
>>>>>>>
>>>>>>> On 4/24/19 11:27 AM, claudius.heine.ext@siemens.com wrote:
>>>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I could not reproduce the raspberrypi issue Maxim found in his 
>>>>>>>> CI build:
>>>>>>>> http://isar-build.org:8080/job/isar_mosipov_develop/91/consoleText
>>>>>>>>
>>>>>>>> This is my build log:
>>>>>>>> http://isar-build.org:8080/job/isar_claudius_ilbers-ci/54/consoleText 
>>>>>>>>
>>>>>>>>
>>>>>>>> Only the module loading fails, which Maxim stated he will look 
>>>>>>>> into.
>>>>>>>>
>>>>>>>> This patchset uses still the 'old' style for refactoring/cleanup 
>>>>>>>> patchsets,
>>>>>>>> until I automated the official patch submission process as 
>>>>>>>> documented by the
>>>>>>>> CONTRIBUTERS policy.
>>>>>>>>
>>>>>>>> You can find the whole patchset in under this url:
>>>>>>>>
>>>>>>>> https://github.com/cmhe/isar/tree/ch/cleanup
>>>>>>>>
>>>>>>>> regards,
>>>>>>>> Claudius
>>>>>>>>
>>>>>>>> changes from v2:
>>>>>>>>   - replaced 'ROOTFS_ARCH' with 'HOST_ARCH' in sdkchroot.bb to 
>>>>>>>> avoid
>>>>>>>>     populate_sdk issue
>>>>>>>>
>>>>>>>> changes from v1:
>>>>>>>>   - fixed typo in commit message
>>>>>>>>
>>>>>>>> Claudius Heine (8):
>>>>>>>>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>>>>>>>>      base.bbclass
>>>>>>>>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to 
>>>>>>>> isar-bootstrap
>>>>>>>>    buildchroot.bbclass: only cross build if HOST_ARCH != 
>>>>>>>> DISTRO_ARCH
>>>>>>>>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>>>>>>>>    bitbake.conf: remove unneeded and differently used variables
>>>>>>>>    image.bbclass: make IMAGE_ROOTFS overwritable
>>>>>>>>    bitbake.conf: set default QEMU_ARCH variables
>>>>>>>>    buildchroot/configscript: make creation of builder uid/gid 
>>>>>>>> idempotent
>>>>>>>>
>>>>>>>>   meta/classes/base.bbclass                          |  9 +++++++++
>>>>>>>>   meta/classes/buildchroot.bbclass                   |  2 +-
>>>>>>>>   meta/classes/image-sdk-extension.bbclass           |  2 +-
>>>>>>>>   meta/classes/image.bbclass                         |  2 +-
>>>>>>>>   meta/classes/isar-bootstrap-helper.bbclass         | 14 
>>>>>>>> --------------
>>>>>>>>   meta/conf/bitbake.conf                             | 13 
>>>>>>>> ++++++++-----
>>>>>>>>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 
>>>>>>>> +++--------
>>>>>>>>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
>>>>>>>>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
>>>>>>>>   .../buildchroot/buildchroot-host.bb                |  1 +
>>>>>>>>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
>>>>>>>>   .../buildchroot/files/configscript.sh              |  4 ++--
>>>>>>>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
>>>>>>>>   13 files changed, 30 insertions(+), 41 deletions(-)
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
> 
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de