From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6467463440282681344 X-Received: by 10.28.142.2 with SMTP id q2mr207781wmd.32.1507114652595; Wed, 04 Oct 2017 03:57:32 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.11.143 with SMTP id 137ls3284333wml.0.canary-gmail; Wed, 04 Oct 2017 03:57:32 -0700 (PDT) X-Google-Smtp-Source: AOwi7QBXd6VuW5LyBaotCQcf/PLY9bkZ2T7Xn3QntQxVToURhj46GVuRFJ5YTDyUgYWKLpqEH+I+ X-Received: by 10.28.17.7 with SMTP id 7mr1450039wmr.25.1507114652328; Wed, 04 Oct 2017 03:57:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507114652; cv=none; d=google.com; s=arc-20160816; b=weZ8/nGKif4OT9cbYqUfBBY/QwgTYMaYfyUS8UJUjJcA+poxsjZGGsV/8E5jQkW6gg EOZ8S54kIq9U0kORjKB3jDoZfkidNjbbeRsMdcBA6Y3p6B9a5kX7SL3VJ231hrYP3ru6 RYK0GdJugWrXyGNtSxI7YlfibNT7VxNKT9iZgjQbRIPOLvE2jk2Uo4k6LxoWyu+aahUM Xm/E9N9HJoxbSb+YDJnjExdYKuWVkE66lmT+j4opa+tGpS/VRsUvnW/Yxw9QyPpz4gx1 V273Nhy6YiguHZY6DU38N3BRsn7ew3QUoXXq83V0F7VDaGhJc9WOsQIeT1PvCPcwX+3K AO3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=ms/QJL0w1UIxaA0DoyN+H2VD3WpuvbKeCJdVnyGflP0=; b=jlmduBWWTktTFQ9M7sWggksYSA24TKlPuK4+7PKlBmSTb6rIwW4qKCN9dDpddsBejm 6ccUS9tPvZFZ/Y0x8hlBt7t2RVn7lOTJMobn1reQfc2UygsS0+rDP3+fqgmNS+jnOaJm U6WlX0A9WuQjpBWNQ4sWzHFuo0IKvYHSaq/uE/PCu1/+UuHf1XRmeRfZZB0qBfYw1O+7 gDVScs04FAezicaJfT0RvBVNHg28VEkm7Mn8n3nNY7aA8194VHs3Rkpzb+Wj7LawXlye V/mHZCcctGlPa/tkNWWPq+32Vf5je3wkpUC6KP2okRn+lDor1Uhf549gqS+ApPH7kjMz iugg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id n82si1237462wmg.3.2017.10.04.03.57.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2017 03:57:32 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id v94AvTRF031521 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 4 Oct 2017 12:57:30 +0200 Subject: Re: [PATCH 2/4] meta-isar-bin: Generate cache repos To: Henning Schild Cc: isar-users@googlegroups.com References: <20171002154531.4930-1-asmirnov@ilbers.de> <20171002154531.4930-3-asmirnov@ilbers.de> <20171004110339.5a80221e@md1em3qc> From: Alexander Smirnov Message-ID: <8bc311b1-22ec-6b14-a70b-82af831fe544@ilbers.de> Date: Wed, 4 Oct 2017 13:57:23 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20171004110339.5a80221e@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 1AEfMlDOWkHZ On 10/04/2017 12:03 PM, Henning Schild wrote: > Am Mon, 2 Oct 2017 18:45:29 +0300 > schrieb Alexander Smirnov : > >> Generate repos for each configuration in multiconfig. >> >> Signed-off-by: Alexander Smirnov >> --- >> meta-isar-bin/conf/layer.conf | 3 +++ >> meta-isar/conf/local.conf.sample | 4 ++++ >> meta/classes/image.bbclass | 35 >> ++++++++++++++++++++++++++++++++++- 3 files changed, 41 >> insertions(+), 1 deletion(-) >> >> diff --git a/meta-isar-bin/conf/layer.conf >> b/meta-isar-bin/conf/layer.conf index 3518184..5ef6e60 100644 >> --- a/meta-isar-bin/conf/layer.conf >> +++ b/meta-isar-bin/conf/layer.conf >> @@ -9,3 +9,6 @@ DEBCACHEDIR ?= "${LAYERDIR}/apt" >> >> # Path to the configuration files templates used by `reprepro` >> DEBFILESDIR ?= "${LAYERDIR}/files" >> + >> +# Path to the databases used by `reprepro` >> +DEBDBDIR ?= "${LAYERDIR}/db" >> diff --git a/meta-isar/conf/local.conf.sample >> b/meta-isar/conf/local.conf.sample index a456b1b..660958f 100644 >> --- a/meta-isar/conf/local.conf.sample >> +++ b/meta-isar/conf/local.conf.sample >> @@ -158,3 +158,7 @@ IMAGE_INSTALL = "hello example-raw" >> # >> # Default parallel jobs for bitbake: >> BB_NUMBER_THREADS = "4" >> + >> +# >> +# Number of attempts to try to get reprepro lock for access to apt >> cache +REPREPRO_LOCK_ATTEMPTS = "16" >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass >> index d30b139..b07775a 100644 >> --- a/meta/classes/image.bbclass >> +++ b/meta/classes/image.bbclass >> @@ -1,5 +1,5 @@ >> # This software is a part of ISAR. >> -# Copyright (C) 2015-2016 ilbers GmbH >> +# Copyright (C) 2015-2017 ilbers GmbH >> >> KERNEL_IMAGE ?= "" >> INITRD_IMAGE ?= "" >> @@ -14,6 +14,30 @@ CACHE_CONF_DIR = "${DEBCACHEDIR}/${DISTRO}/conf" >> do_cache_config[dirs] = "${CACHE_CONF_DIR}" >> do_cache_config[stamp-extra-info] = "${DISTRO}" >> >> +call_reprepro() { >> + for i in $(seq 1 ${REPREPRO_LOCK_ATTEMPTS}); do >> + # According to `sh` manual page, shell exit statuses fall >> between >> + # 0-255. The EEXIST error code is (-17), so casting to >> usigned 8-bit > > cant you use python to get that number and use the name "EEXIST" instead > of a hardcoded number? I think i sent a sniplet in an earlier mail. No we can't. In python I've checked the code EEXIST, and it is (17), while return value from subprocess call is still (239). So using python here is only overhead in correct wrapping of reprepro call. > >> + # integer results value (239). >> + eexist="239" >> + retval="0" >> + reprepro --waitforlock 1 $* || retval="$?" > > The waitforlock should be skipped altogether, which means try it once. > The current version means try twice. I got that wrong when i suggested > to go for "1", sorry. 'waitforlock 1' adds 10 seconds delay between attempts. So the idea to use this form to avoid extra sleep calls with magic timeout value. > >> + # If reprepro has failed to get database lock, it returns >> EEXIST code. >> + # In this case we continue trying to get lock until max >> amount of >> + # attempts is reached. >> + if [ $retval -eq $eexist ]; then >> + bbwarn "Failed to get reprepro lock, trying again..." > > Are there other cases where reprepro returns -EEXIST? > AFAIK there is only one case when reprepro could return EEXIST - it's morgue feature which is not used here. All the other EEXIST checks are processed internally. Alex > Henning > >> + else >> + break >> + fi >> + done >> + >> + if [ $retval -ne 0 ]; then >> + bbfatal "reprepro failed" >> + fi >> +} >> + >> # Generate reprepro config for current distro if it doesn't exist. >> Once it's # generated, this task should do nothing. >> do_cache_config() { >> @@ -21,6 +45,15 @@ do_cache_config() { >> sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ >> ${DEBFILESDIR}/distributions.in > >> ${CACHE_CONF_DIR}/distributions fi >> + >> + path_cache="${DEBCACHEDIR}/${DISTRO}" >> + path_databases="${DEBDBDIR}/${DISTRO}" >> + >> + if [ ! -d "${path_databases}" ]; then >> + call_reprepro -b ${path_cache} \ >> + --dbdir ${path_databases} \ >> + export ${DEBDISTRONAME} >> + fi >> } >> >> addtask cache_config before do_fetch >