Project isar_claudius_ilbers_fast_ci on isar-build.org

Project isar_claudius_ilbers_fast_ci on isar-build.org

[Maxim Yu. Osipov]

Hi Claudius,

I've created and configured the project Project 
isar_claudius_ilbers_fast_ci to launch "fast" CI.

I've started the test build in this project (it clones your branch 
ch/ilbers-ci from https://github.com/cmhe/isar)and it immediately 
reproduced the error I reported before:

http://isar-build.org:8080/job/isar_claudius_ilbers_fast_ci/1/console

Any problems with access,
please let me know.

Maxim.

-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: Project isar_claudius_ilbers_fast_ci on isar-build.org

[Jan Kiszka]

On 25.04.19 11:50, Maxim Yu. Osipov wrote:
> Hi Claudius,
> 
> I've created and configured the project Project isar_claudius_ilbers_fast_ci to 
> launch "fast" CI.
> 
> I've started the test build in this project (it clones your branch ch/ilbers-ci 
> from https://github.com/cmhe/isar)and it immediately reproduced the error I 
> reported before:
> 
> http://isar-build.org:8080/job/isar_claudius_ilbers_fast_ci/1/console
> 
> Any problems with access,
> please let me know.

In fact, it didn't even need that complication in the first place: The issue is 
also reproducible in our gitlab-ci environment.

Claudius, you have access to it, and you can look into the logs what went wrong 
or push new versions there. It just takes a few minutes to get results as the 
failure is early.

Once we understood this issue, it would also be good to understand why Claudius 
earlier run on the jenkins CI didn't reveal it. But one after the other.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: Project isar_claudius_ilbers_fast_ci on isar-build.org

[Claudius Heine]

Hi Jan,

On 25/04/2019 12.19, Jan Kiszka wrote:
> On 25.04.19 11:50, Maxim Yu. Osipov wrote:
>> Hi Claudius,
>>
>> I've created and configured the project Project 
>> isar_claudius_ilbers_fast_ci to launch "fast" CI.
>>
>> I've started the test build in this project (it clones your branch 
>> ch/ilbers-ci from https://github.com/cmhe/isar)and it immediately 
>> reproduced the error I reported before:
>>
>> http://isar-build.org:8080/job/isar_claudius_ilbers_fast_ci/1/console
>>
>> Any problems with access,
>> please let me know.
> 
> In fact, it didn't even need that complication in the first place: The 
> issue is also reproducible in our gitlab-ci environment.
> 
> Claudius, you have access to it, and you can look into the logs what 
> went wrong or push new versions there. It just takes a few minutes to 
> get results as the failure is early.
> 
> Once we understood this issue, it would also be good to understand why 
> Claudius earlier run on the jenkins CI didn't reveal it. But one after 
> the other.

Well probably because "fast" enabled cross-compilation.

Now I enabled cross-compilation for rpi-stretch locally and now I cannot 
build next there, but it builds apparently on isar-build...

I would really love to be able to reproduce exactly that issue locally...

Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: Project isar_claudius_ilbers_fast_ci on isar-build.org

[Jan Kiszka]

On 25.04.19 12:24, Claudius Heine wrote:
> Hi Jan,
> 
> On 25/04/2019 12.19, Jan Kiszka wrote:
>> On 25.04.19 11:50, Maxim Yu. Osipov wrote:
>>> Hi Claudius,
>>>
>>> I've created and configured the project Project isar_claudius_ilbers_fast_ci 
>>> to launch "fast" CI.
>>>
>>> I've started the test build in this project (it clones your branch 
>>> ch/ilbers-ci from https://github.com/cmhe/isar)and it immediately reproduced 
>>> the error I reported before:
>>>
>>> http://isar-build.org:8080/job/isar_claudius_ilbers_fast_ci/1/console
>>>
>>> Any problems with access,
>>> please let me know.
>>
>> In fact, it didn't even need that complication in the first place: The issue 
>> is also reproducible in our gitlab-ci environment.
>>
>> Claudius, you have access to it, and you can look into the logs what went 
>> wrong or push new versions there. It just takes a few minutes to get results 
>> as the failure is early.
>>
>> Once we understood this issue, it would also be good to understand why 
>> Claudius earlier run on the jenkins CI didn't reveal it. But one after the other.
> 
> Well probably because "fast" enabled cross-compilation.
> 
> Now I enabled cross-compilation for rpi-stretch locally and now I cannot build 
> next there, but it builds apparently on isar-build...

...and apparently also on gitlab-ci. So we have a race that probably papers over 
it in larger CI runs. If you can reproduce without that race, please debug.

> 
> I would really love to be able to reproduce exactly that issue locally...

Then buy yourself a blade server as laptop ;). Look, not all issues can be 
reproduced locally, at least not before you understood the boundary conditions. 
That's very common and not always the problem of some CI system.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: Project isar_claudius_ilbers_fast_ci on isar-build.org

[Baurzhan Ismagulov]

On Thu, Apr 25, 2019 at 12:30:58PM +0200, Jan Kiszka wrote:
> > Now I enabled cross-compilation for rpi-stretch locally and now I cannot
> > build next there, but it builds apparently on isar-build...
> 
> ...and apparently also on gitlab-ci.

Do I understand correctly, next does build on gitlab-ci?


> So we have a race that probably papers
> over it in larger CI runs. If you can reproduce without that race, please
> debug.

That might also be a simpler issue like missing host packages. rpi requires
some packages that are not required by other targets. If one doesn't usually
build for rpi, those might be missing.


> > I would really love to be able to reproduce exactly that issue locally...
> 
> Then buy yourself a blade server as laptop ;). Look, not all issues can be
> reproduced locally, at least not before you understood the boundary
> conditions. That's very common and not always the problem of some CI system.

next should build locally. If there is a problem with the local build, we
shouldn't ignore it. Maybe after this first problem is solved, one can also
reproduce exactly that issue locally.


With kind regards,
Baurzhan.

Re: Project isar_claudius_ilbers_fast_ci on isar-build.org

[Jan Kiszka]

On 25.04.19 12:54, Baurzhan Ismagulov wrote:
> On Thu, Apr 25, 2019 at 12:30:58PM +0200, Jan Kiszka wrote:
>>> Now I enabled cross-compilation for rpi-stretch locally and now I cannot
>>> build next there, but it builds apparently on isar-build...
>>
>> ...and apparently also on gitlab-ci.
> 
> Do I understand correctly, next does build on gitlab-ci?
> 

It's still running, but it seems to have past the failure points with Claudius' 
patches applied.

> 
>> So we have a race that probably papers
>> over it in larger CI runs. If you can reproduce without that race, please
>> debug.
> 
> That might also be a simpler issue like missing host packages. rpi requires
> some packages that are not required by other targets. If one doesn't usually
> build for rpi, those might be missing.
> 

There are multiple issues, only one of them is in the rpi target. There is 
generally no way around studying the logs in details.

> 
>>> I would really love to be able to reproduce exactly that issue locally...
>>
>> Then buy yourself a blade server as laptop ;). Look, not all issues can be
>> reproduced locally, at least not before you understood the boundary
>> conditions. That's very common and not always the problem of some CI system.
> 
> next should build locally. If there is a problem with the local build, we
> shouldn't ignore it. Maybe after this first problem is solved, one can also
> reproduce exactly that issue locally.

Let's understand the problems first, then do the post-portem analysis why there 
weren't found early if they could have.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: Project isar_claudius_ilbers_fast_ci on isar-build.org

[Baurzhan Ismagulov]

On Thu, Apr 25, 2019 at 12:58:13PM +0200, Jan Kiszka wrote:
> It's still running, but it seems to have past the failure points with
> Claudius' patches applied.

Ok, thanks for the confirmation.


> > next should build locally. If there is a problem with the local build, we
> > shouldn't ignore it. Maybe after this first problem is solved, one can also
> > reproduce exactly that issue locally.
> 
> Let's understand the problems first, then do the post-portem analysis why
> there weren't found early if they could have.

My point was to ease Claudius's debugging rather than post-mortem analysis. I
agree that one should read logs, too; I just happen to be more productive
locally.


With kind regards,
Baurzhan.