Re: [PATCH v3 2/4] meta-isar-bin: Generate cache repos

[Henning Schild <henning.schild@siemens.com>]

Am Mon, 25 Sep 2017 12:59:58 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> Generate repos for each configuration in multiconfig.
> 
> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> ---
>  meta-isar-bin/conf/layer.conf    |  3 +++
>  meta-isar/conf/local.conf.sample |  4 ++++
>  meta/classes/image.bbclass       | 16 ++++++++++++++++
>  3 files changed, 23 insertions(+)
> 
> diff --git a/meta-isar-bin/conf/layer.conf
> b/meta-isar-bin/conf/layer.conf index 3518184..5ef6e60 100644
> --- a/meta-isar-bin/conf/layer.conf
> +++ b/meta-isar-bin/conf/layer.conf
> @@ -9,3 +9,6 @@ DEBCACHEDIR ?= "${LAYERDIR}/apt"
>  
>  # Path to the configuration files templates used by `reprepro`
>  DEBFILESDIR ?= "${LAYERDIR}/files"
> +
> +# Path to the databases used by `reprepro`
> +DEBDBDIR ?= "${LAYERDIR}/db"
> diff --git a/meta-isar/conf/local.conf.sample
> b/meta-isar/conf/local.conf.sample index a456b1b..660958f 100644
> --- a/meta-isar/conf/local.conf.sample
> +++ b/meta-isar/conf/local.conf.sample
> @@ -158,3 +158,7 @@ IMAGE_INSTALL = "hello example-raw"
>  #
>  # Default parallel jobs for bitbake:
>  BB_NUMBER_THREADS = "4"
> +
> +#
> +# Number of attempts to try to get reprepro lock for access to apt
> cache +REPREPRO_LOCK_ATTEMPTS = "16"
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 9ab9b19..6f39c7a 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -14,11 +14,27 @@ CACHE_CONF_DIR = "${DEBCACHEDIR}/${DISTRO}/conf"
>  do_cache_config[dirs] = "${CACHE_CONF_DIR}"
>  do_cache_config[stamp-extra-info] = "${DISTRO}"
>  
> +call_reprepro() {
> +    for i in $(seq 1 ${REPREPRO_LOCK_ATTEMPTS}); do
> +        reprepro --waitforlock 5 $* \
> +        || bbwarn "Failed to get reprepro lock, trying again..."

Now we have two loops and the number from above need to be bultiplied
with the inner number .. 5*16 times 10s. I still do not see how that
solves the problem. But i can imagine those two loops really slowing
down builds that fail. Looks like they potentially add 800s build-time.
And every error of reprepro is turned into "Failed to get lock" ... As
far as i looked at reprepro it should return EEXIST in the
contention case.

Henning

> +    done
> +}
> +
>  do_cache_config() {
>      if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then
>          sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \
>              ${DEBFILESDIR}/distributions.in >
> ${CACHE_CONF_DIR}/distributions fi
> +
> +    path_cache="${DEBCACHEDIR}/${DISTRO}"
> +    path_databases="${DEBDBDIR}/${DISTRO}"
> +
> +    if [ ! -d "${path_databases}" ]; then
> +        call_reprepro -b ${path_cache} \
> +                      --dbdir ${path_databases} \
> +                      export ${DEBDISTRONAME}
> +    fi
>  }
>  
>  addtask cache_config before do_fetch