From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6467463440282681344 X-Received: by 10.28.150.130 with SMTP id y124mr2372063wmd.22.1507126208142; Wed, 04 Oct 2017 07:10:08 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.168.14 with SMTP id r14ls3328916wme.3.canary-gmail; Wed, 04 Oct 2017 07:10:07 -0700 (PDT) X-Google-Smtp-Source: AOwi7QCTlfwWcgEzpXvApsAyJazdBCBZLg0XomR8ejLF4HFv0rfPS9ZeGiE0uer54+CSiAPO6Agv X-Received: by 10.28.184.78 with SMTP id i75mr2886541wmf.30.1507126207784; Wed, 04 Oct 2017 07:10:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507126207; cv=none; d=google.com; s=arc-20160816; b=fUAjNO3JYCdFgUTHJr4HC/dKLr+K3zBhtBu9c/65JO+tmmgdeNSLmBPvrt0CKG+9rM mEvKFzV8hCaBaXevglew75xXCQKrSj5WwQuF25jh3IG/LnH6VtaHVu52wOKIyCwvFAyl /NF9vga7n2esZ7Y2toRqxWu5yQXTEMHMfRTZ0GhUPWxNhvZmvvHJooDaaQuqFn5jCrH8 fBxrVeOkIMKLH/xhYabOryKo/onZ0g/rc06Z0p+e6CCd2m/oq4rd0ZoNBFDrxKxX7S9h UTE+mLEBOYNM4idfs4vyvDpmNe/YSlG/NkDHSr5PG6BSxaleZly/rAlZ6h4yY/CXRVDA Ogxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=rPY+XdZsqpyetTdxXdlksbQLTtHQlaPJG1tVW9cSAAE=; b=meiKqG1lXBB8z3E0imBBtZY73ACKfsJZChFRMnlNjsDUp0ulJFIb7daqCZH8ovFD2I tncyVOT4zDT2G20Y9HBKldMK4Jla33NOTVkcxqlgJiIqIxiQjsiuyFhgwV72BufgjPPG NifvOBQXMiNXINmopuuxLwrtgmBluBK9BzGvQQSfy1jGD0JAUVN2xUmay+vOKs2wPBlC qqKyqqya8C67TCyhf3OnyqiTo7Zh2xv1lZYQPp/M8qK36TjAweBaLnFKYkTVW/XdfRaP H48cGGQzcy4XZBq2ehVzb+jyPGwGk+ayGGKkwSugqWc4G7JEMHxIlLtQyXoojo+aBgls pNRw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.14 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id 196si1124462wmk.4.2017.10.04.07.10.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2017 07:10:07 -0700 (PDT) Received-SPF: neutral (google.com: 192.35.17.14 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.14 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id v94EA7po003302 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 4 Oct 2017 16:10:07 +0200 Received: from md1em3qc ([139.22.127.98]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id v94EA3Yr015605; Wed, 4 Oct 2017 16:10:05 +0200 Date: Wed, 4 Oct 2017 16:09:33 +0200 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH 2/4] meta-isar-bin: Generate cache repos Message-ID: <20171004160933.7e09a3bd@md1em3qc> In-Reply-To: <8bc311b1-22ec-6b14-a70b-82af831fe544@ilbers.de> References: <20171002154531.4930-1-asmirnov@ilbers.de> <20171002154531.4930-3-asmirnov@ilbers.de> <20171004110339.5a80221e@md1em3qc> <8bc311b1-22ec-6b14-a70b-82af831fe544@ilbers.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: Ls4qOYSkQfps Am Wed, 4 Oct 2017 13:57:23 +0300 schrieb Alexander Smirnov : > On 10/04/2017 12:03 PM, Henning Schild wrote: > > Am Mon, 2 Oct 2017 18:45:29 +0300 > > schrieb Alexander Smirnov : > > > >> Generate repos for each configuration in multiconfig. > >> > >> Signed-off-by: Alexander Smirnov > >> --- > >> meta-isar-bin/conf/layer.conf | 3 +++ > >> meta-isar/conf/local.conf.sample | 4 ++++ > >> meta/classes/image.bbclass | 35 > >> ++++++++++++++++++++++++++++++++++- 3 files changed, 41 > >> insertions(+), 1 deletion(-) > >> > >> diff --git a/meta-isar-bin/conf/layer.conf > >> b/meta-isar-bin/conf/layer.conf index 3518184..5ef6e60 100644 > >> --- a/meta-isar-bin/conf/layer.conf > >> +++ b/meta-isar-bin/conf/layer.conf > >> @@ -9,3 +9,6 @@ DEBCACHEDIR ?= "${LAYERDIR}/apt" > >> > >> # Path to the configuration files templates used by `reprepro` > >> DEBFILESDIR ?= "${LAYERDIR}/files" > >> + > >> +# Path to the databases used by `reprepro` > >> +DEBDBDIR ?= "${LAYERDIR}/db" > >> diff --git a/meta-isar/conf/local.conf.sample > >> b/meta-isar/conf/local.conf.sample index a456b1b..660958f 100644 > >> --- a/meta-isar/conf/local.conf.sample > >> +++ b/meta-isar/conf/local.conf.sample > >> @@ -158,3 +158,7 @@ IMAGE_INSTALL = "hello example-raw" > >> # > >> # Default parallel jobs for bitbake: > >> BB_NUMBER_THREADS = "4" > >> + > >> +# > >> +# Number of attempts to try to get reprepro lock for access to apt > >> cache +REPREPRO_LOCK_ATTEMPTS = "16" > >> diff --git a/meta/classes/image.bbclass > >> b/meta/classes/image.bbclass index d30b139..b07775a 100644 > >> --- a/meta/classes/image.bbclass > >> +++ b/meta/classes/image.bbclass > >> @@ -1,5 +1,5 @@ > >> # This software is a part of ISAR. > >> -# Copyright (C) 2015-2016 ilbers GmbH > >> +# Copyright (C) 2015-2017 ilbers GmbH > >> > >> KERNEL_IMAGE ?= "" > >> INITRD_IMAGE ?= "" > >> @@ -14,6 +14,30 @@ CACHE_CONF_DIR = "${DEBCACHEDIR}/${DISTRO}/conf" > >> do_cache_config[dirs] = "${CACHE_CONF_DIR}" > >> do_cache_config[stamp-extra-info] = "${DISTRO}" > >> > >> +call_reprepro() { > >> + for i in $(seq 1 ${REPREPRO_LOCK_ATTEMPTS}); do > >> + # According to `sh` manual page, shell exit statuses fall > >> between > >> + # 0-255. The EEXIST error code is (-17), so casting to > >> usigned 8-bit > > > > cant you use python to get that number and use the name "EEXIST" > > instead of a hardcoded number? I think i sent a sniplet in an > > earlier mail. > > No we can't. In python I've checked the code EEXIST, and it is (17), > while return value from subprocess call is still (239). So using > python here is only overhead in correct wrapping of reprepro call. Just do the math in python, skip the long comment and get rid of the magic number. python -c "import errno; print(256-errno.EEXIST)" > > > >> + # integer results value (239). > >> + eexist="239" > >> + retval="0" > >> + reprepro --waitforlock 1 $* || retval="$?" > > > > The waitforlock should be skipped altogether, which means try it > > once. The current version means try twice. I got that wrong when i > > suggested to go for "1", sorry. > > 'waitforlock 1' adds 10 seconds delay between attempts. So the idea > to use this form to avoid extra sleep calls with magic timeout value. Now you have two nested loops again and the name REPREPRO_LOCK_ATTEMPTS is wrong, because you do it 2*REPREPRO_LOCK_ATTEMPTS. Either divide that number by two or introduce a sleep on the error-path. > > > >> + # If reprepro has failed to get database lock, it returns > >> EEXIST code. > >> + # In this case we continue trying to get lock until max > >> amount of > >> + # attempts is reached. > >> + if [ $retval -eq $eexist ]; then > >> + bbwarn "Failed to get reprepro lock, trying > >> again..." > > > > Are there other cases where reprepro returns -EEXIST? > > > > AFAIK there is only one case when reprepro could return EEXIST - it's > morgue feature which is not used here. All the other EEXIST checks > are processed internally. Ok, just wanted to make sure someone checked that. Henning > Alex > > > Henning > > > >> + else > >> + break > >> + fi > >> + done > >> + > >> + if [ $retval -ne 0 ]; then > >> + bbfatal "reprepro failed" > >> + fi > >> +} > >> + > >> # Generate reprepro config for current distro if it doesn't > >> exist. Once it's # generated, this task should do nothing. > >> do_cache_config() { > >> @@ -21,6 +45,15 @@ do_cache_config() { > >> sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ > >> ${DEBFILESDIR}/distributions.in > > >> ${CACHE_CONF_DIR}/distributions fi > >> + > >> + path_cache="${DEBCACHEDIR}/${DISTRO}" > >> + path_databases="${DEBDBDIR}/${DISTRO}" > >> + > >> + if [ ! -d "${path_databases}" ]; then > >> + call_reprepro -b ${path_cache} \ > >> + --dbdir ${path_databases} \ > >> + export ${DEBDISTRONAME} > >> + fi > >> } > >> > >> addtask cache_config before do_fetch > >