Re: Another CI splat: do_cache_base_repo: Failed

["Maxim Yu. Osipov" <mosipov@ilbers.de>]

Hi Cedric,

Thank you for fast feedback.

I agree with you - race may happen in populate_base_repo.

Your patch is very welcome.

Thanks,
Maxim.

On 2/7/19 2:22 PM, Hombourger, Cedric wrote:
> 
> Don't we need to add:
> 
> do_cache_base_repo[lockfiles] = "${REPO_BASE_DIR}/isar.lock"
> 
> Your CI is building multiple images in parallel, this means that we will end-up having multiple processes populate the base apt cache concurrently
> We'd therefore need to serialize accesses to the repo
> 
> I am happy to generate a patch if you believe my theory makes sense
> 
> Cedric
> 
> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
> Sent: Wednesday, January 16, 2019 5:29 AM
> To: isar-users <isar-users@googlegroups.com>; Hombourger, Cedric <Cedric_Hombourger@mentor.com>; Maksim Osipov <mosipov@ilbers.de>
> Subject: Re: Another CI splat: do_cache_base_repo: Failed
> 
> On 07.01.19 16:00, [ext] Jan Kiszka wrote:
>> Hi,
>>
>> Familiar (even if rare)?
>>
>> NOTE: recipe isar-image-base-1.0-r0: task do_cache_base_repo: Started
>> ERROR: mc:qemuarm-stretch:isar-image-base-1.0-r0 do_cache_base_repo:
>> Function
>> failed: do_cache_base_repo (log file is located at
>> /builds/ebsy/debian/isar/build/tmp/work/debian-stretch-armhf/isar-imag
>> e-base/temp/log.do_cache_base_repo.55127)
>>
>> ERROR: Logfile of failure stored in:
>> /builds/ebsy/debian/isar/build/tmp/work/debian-stretch-armhf/isar-imag
>> e-base/temp/log.do_cache_base_repo.55127
>>
>> Log data follows:
>> | DEBUG: Executing shell function do_cache_base_repo The lock file
>> '/builds/ebsy/debian/isar/build/downloads/base-apt/db/debian/lockfile'
>> already exists. There might be another instance with the
>> | same database dir running. To avoid locking overhead, only one
>> | process can access the database at the same time. Do not delete the
>> | lock file unless you are sure no other version is still running!
>> | There have been errors!
>> | WARNING: exit code 239 from a shell command.
>> | ERROR: Function failed: do_cache_base_repo (log file is located at
>> /builds/ebsy/debian/isar/build/tmp/work/debian-stretch-armhf/isar-imag
>> e-base/temp/log.do_cache_base_repo.55127)
>>
>> NOTE: recipe isar-image-base-1.0-r0: task do_cache_base_repo: Failed
>> ERROR: Task
>> (multiconfig:qemuarm-stretch:/builds/ebsy/debian/isar/meta-isar/recipe
>> s-core/images/isar-image-base.bb:do_cache_base_repo)
>> failed with exit code '1'
>>
>>
>> Cedric, you once sent a related patch ("dpkg: acquire lock when
>> calling
>> reprepro") that seems like it was never merged. Was that related to a
>> similar error?
> 
> Happened again yesterday, and even the retry failed now. Seems we have at least a better chance to reproduce this... Any comments on this bug?
> 
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov