Re: Build can sporadically fail due to dpkg contention in buildchroot

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/11/2018 09:18 PM, Jan Kiszka wrote:
> On 2018-02-11 17:55, Alexander Smirnov wrote:
>> Hi,
>>
>> On 02/11/2018 06:17 PM, Jan Kiszka wrote:
>>> Hi all,
>>>
>>> I got this failure of example-hello:do_build already twice while doing
>>> rebuild tests with my kernel series (ie. with more independent
>>> buildchroot users):
>>>
>>> DEBUG: Executing shell function do_build
>>> Get:1 file:/isar-apt isar InRelease
>>> Ign:1 file:/isar-apt isar InRelease
>>> Get:2 file:/isar-apt isar Release [2,864 B]
>>> Get:2 file:/isar-apt isar Release [2,864 B]
>>> Get:3 file:/isar-apt isar Release.gpg
>>> Ign:3 file:/isar-apt isar Release.gpg
>>> Get:4 file:/isar-apt isar/main amd64 Packages [1,135 B]
>>> Reading package lists...
>>> W: The repository 'file:/isar-apt isar Release' is not signed.
>>> hostname: No address associated with hostname
>>> dh_testdir
>>> dh_testroot
>>> dh_prep
>>> dh_testdir
>>> dh_testroot
>>> dh_install
>>> dh_install: Compatibility levels before 9 are deprecated (level 7 in use)
>>> dh_installdocs
>>> dh_installdocs: Compatibility levels before 9 are deprecated (level 7
>>> in use)
>>> dh_installchangelogs
>>> dh_compress
>>> dh_fixperms
>>> dh_installdeb
>>> dh_installdeb: Compatibility levels before 9 are deprecated (level 7
>>> in use)
>>> dh_gencontrol
>>> dh_md5sums
>>> dh_builddeb
>>> dpkg-deb: building package 'hello-build-deps' in
>>> '../hello-build-deps_0.2_all.deb'.
>>>
>>
>> Good catch!
>>
>>> The package has been created.
>>> Attention, the package has been created in the current directory,
>>> not in ".." as indicated by the message above!
>>> dpkg: error: dpkg status database is locked by another process
>>> mk-build-deps: dpkg --unpack failed
>>> [...]
>>>
>>> So we have a concurrency problem when building over the same dpkg
>>> database. Looks like we need to synchronize (lock-protect) the access to
>>> it, which also means pulling out the dependency installation from the
>>> regular build step. Is that feasible at all? Any alternatives (besides
>>> retrying such builds...)?
>>
>> In general we could do this easily:
>>
>> 1. Split the content of build.sh into two functions, for example:
>>   - install_build_deps
>>   - build_package
>>
>> 2. Spit the bitbake do_build() into two tasks:
>>
>> do_install_build_deps() {
>>      ... build.sh install_build_deps ...
>> }
>>
>> addtask install_build_deps before do_build after do_unpack
>>
>> do_build_package() {
>>      ... build.sh build_package ...
>> }
>>
>> 3. Using bitbake synchronization primitives, protect the fist task from
>> parallel execution.
>>
>> If you are OK with this, I could do this tomorrow.
> 
> I'm still concerned how well this will scale:
> 
> a) We have additional users we already know of (linux-kernel.bbclass).
>     We will need to provide them the same means.
> 
> b) There might be more users hidden in today's or future recipes...
> 

Now we have pipeline:
  - do_fetch, do_unpack, do_build.

I propose to extend this pipeline by one extra task:
  - do_fetch, do_unpack, do_install_build_deps, do_build.

These are the core tasks that have default payload defined in calsses, 
so you should not touch them in custom recipes.

For better scale-ability, a separate class could be created:

8<--

dpkg-build-deps.bbclass:

do_install_build_deps() {
    ... call mk-build-deps
}

addtask install_build_deps before do_build after do_unpack

8<--

So if you want this functionality in your class, for example in 
dpkg.bbclass, so just include it. I can't imagine if we will have so 
many different classes to build something...

Alex