Re: [PATCH v2 2/4] meta-isar-bin: Generate cache repos

[Alexander Smirnov <asmirnov@ilbers.de>]

On 09/20/2017 11:11 AM, Henning Schild wrote:
> Am Tue, 19 Sep 2017 15:20:50 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> 
>> Generate repos for each configuration in multiconfig.
>>
>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>> ---
>>   meta-isar-bin/conf/layer.conf |  3 +++
>>   meta/classes/image.bbclass    | 14 ++++++++++++++
>>   2 files changed, 17 insertions(+)
>>
>> diff --git a/meta-isar-bin/conf/layer.conf
>> b/meta-isar-bin/conf/layer.conf index 3518184..5ef6e60 100644
>> --- a/meta-isar-bin/conf/layer.conf
>> +++ b/meta-isar-bin/conf/layer.conf
>> @@ -9,3 +9,6 @@ DEBCACHEDIR ?= "${LAYERDIR}/apt"
>>   
>>   # Path to the configuration files templates used by `reprepro`
>>   DEBFILESDIR ?= "${LAYERDIR}/files"
>> +
>> +# Path to the databases used by `reprepro`
>> +DEBDBDIR ?= "${LAYERDIR}/db"
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 30f241a..9db7371 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -35,6 +35,20 @@ python __anonymous () {
>>           f.close()
>>   }
>>   
>> +do_create_cache_db[stamp-extra-info] = "${DISTRO}"
>> +
>> +do_create_cache_db() {
>> +    path_cache="${DEBCACHEDIR}/${DISTRO}"
>> +    path_databases="${DEBDBDIR}/${DISTRO}"
>> +
>> +    if [ ! -d "${path_databases}" ]; then
>> +        reprepro --waitforlock 3 -b ${path_cache} --dbdir
>> ${path_databases} \
>> +                 export ${DEBDISTRONAME}
>> +    fi
>> +}
>> +
>> +addtask create_cache_db before do_fetch
> 
> This adds a new tool dependency, maybe update docs with the patch or
> keep in mind for later.

For sure, I'll document everything before merge, when we agree on 
general architecture.

> 
> Why "waitforlock 3" is there any chance at all that multiple images run
> into that at the same time? And if so how do we know 3 is enough? I just
> tested that a second "reprepro export" will still return 0, so it seems
> safe to do that multiple times, not sure what would happen if some
> packages where already added and you "export" again.

Yes, this task could be run multiple times, for example by the following 
line:

bitbake \
   multiconfig:qemuarm-jessie:isar-image-base \
   multiconfig:qemui386-jessie:isar-image-base \
   multiconfig:qemuamd64-jessie:isar-image-base

I'll start jessie build for 3 architectures:
  - arm
  - i386
  - amd64

After build, there will be the following in the cache (example for hello):

$ ls -1 meta-isar-bin/apt/debian-jessie/pool/main/h/hello/
hello_0.1+g7f35942-1_amd64.deb
hello_0.1+g7f35942-1_armhf.deb
hello_0.1+g7f35942-1_i386.deb

So using waitforlock is an attempt to protect reprepro from crashing. In 
general this will never happen for single-distro single-arch builds.

Alex

> 
> Henning
> 
>>   do_populate[stamp-extra-info] = "${DISTRO}-${MACHINE}"
>>   
>>   # Install Debian packages, that were built from sources
> 

-- 
With best regards,
Alexander Smirnov

ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov