From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6522497667596025856 X-Received: by 10.25.161.212 with SMTP id k203mr9097lfe.5.1518637330318; Wed, 14 Feb 2018 11:42:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.78.215 with SMTP id u84ls446791lfk.13.gmail; Wed, 14 Feb 2018 11:42:09 -0800 (PST) X-Google-Smtp-Source: AH8x227kqdhnUPBlujDcuAzX2PowDKcIUvG/n33e6WIVLDNrgNbphlKSeQxvfNiSP3qZGXRZoK0s X-Received: by 10.25.163.211 with SMTP id m202mr8429lfe.23.1518637329681; Wed, 14 Feb 2018 11:42:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518637329; cv=none; d=google.com; s=arc-20160816; b=BHaJp77q9BcevEuf8UHrf0DzDOMB3Rh12qS+SvK4ck8hOtAQBh9G1MgHMEVasE5j1y DAF+Io+jB3o7F1RzdLAP8XStYQCTU8dGEvAPA38+PbST6tfWaFyNL5IdIBGAqAoJbIzB U8ptvPviWk3q99ic9e0aaJCBZKs7fLK5SVTIjeqyusw9bbxgB5ZB3J2fjyQEyYJs7H+B jHJmFQrrclb1D/UPKgmLhydRDtIunzhC/6PbIBf4hFAxiBf/ZgjDZwmPtSQB7aqEg0RW kS1fhPiC5SoxaduskkPKd9yXInSMoqsptfocH69oRpHeo7cLX5EcsUmuWHtebSM4mG2N qHeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:mime-version:user-agent :date:message-id:subject:from:cc:to:arc-authentication-results; bh=4uMuZYVR6F9lszem1S2493nMDJP9zPilO6fuSFnzu94=; b=KDe/joqxiC9fnJxJCn1FnFUYDsVmFfkrYLmLS4+amb7E/CChIwyIifMig8fe9l71iL HENapY8Tuwsm66yoIyjSGYhuh947FxotI742DAFAkqwehhjBH2xcutz7BOFZLsAq4NGg KnX+cdB5L5o3NdpEvuLc9mNlIFSgm/KR5J4emw8i5nopIBoMbHeLCglAP7az6MzZKqZG O9bqPJ4ysW6o3Bihy2nN/IXpOZqZ053yEh/RYPfRz6Qui75rNRkUjYLphUblhqEJmEQj LXysfIgiN55CMkWN51AJ6FAdelrep9IeuLLgafv7jqS86qjYGYpiRZXGSXRFgZlPT47d Btog== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id o26si729137ljc.5.2018.02.14.11.42.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Feb 2018 11:42:09 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w1EJg8xs028004 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Feb 2018 20:42:08 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w1EJg8mL019044; Wed, 14 Feb 2018 20:42:08 +0100 To: isar-users , Alexander Smirnov Cc: "Larson, Chris" From: Jan Kiszka Subject: [PATCH] buildchroot: Lock-protect apt/dpkg operations Message-ID: <92b096db-6db3-401c-0bba-09a31cff2376@siemens.com> Date: Wed, 14 Feb 2018 20:42:08 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: KoXnwL3N4zoy From: Jan Kiszka When running multiple build targets in parallel, they will compete for access on the package database while the related commands do not wait but fail immediately. Signed-off-by: Jan Kiszka --- Fixes parallel builds (CI setup) for me. I just recalled that Chris was reporting the same issue a long time ago, but his suggestion to lock down whole do_build was not that well received... meta/recipes-devtools/buildchroot/files/build.sh | 32 +++++++++++++++--------- 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh index 023a6d6..048f76f 100644 --- a/meta/recipes-devtools/buildchroot/files/build.sh +++ b/meta/recipes-devtools/buildchroot/files/build.sh @@ -2,17 +2,9 @@ # # This software is a part of ISAR. # Copyright (C) 2015-2017 ilbers GmbH +# Copyright (c) 2018 Siemens AG -# Make sure that we have latest isar-apt content. -# Options meaning: -# Dir::Etc::sourcelist - specifies which source to be used -# Dir::Etc::sourceparts - disables looking for the other sources -# APT::Get::List-Cleanup - do not erase obsolete packages list for -# upstream in '/var/lib/apt/lists' -apt-get update \ - -o Dir::Etc::sourcelist="sources.list.d/multistrap-isar-apt.list" \ - -o Dir::Etc::sourceparts="-" \ - -o APT::Get::List-Cleanup="0" +set -e # Go to build directory cd $1 @@ -26,8 +18,24 @@ install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y # Allow unauthenticated feeds install_cmd="${install_cmd} --allow-unauthenticated" -# Install all build deps -mk-build-deps -t "${install_cmd}" -i -r debian/control +( + # Lock-protected because apt and dpkg do not wait in case of contention + flock 42 || exit 1 + + # Make sure that we have latest isar-apt content. + # Options meaning: + # Dir::Etc::sourcelist - specifies which source to be used + # Dir::Etc::sourceparts - disables looking for the other sources + # APT::Get::List-Cleanup - do not erase obsolete packages list for + # upstream in '/var/lib/apt/lists' + apt-get update \ + -o Dir::Etc::sourcelist="sources.list.d/multistrap-isar-apt.list" \ + -o Dir::Etc::sourceparts="-" \ + -o APT::Get::List-Cleanup="0" + + # Install all build deps + mk-build-deps -t "${install_cmd}" -i -r debian/control +) 42>/dpkg.lock # If autotools files have been created, update their timestamp to # prevent them from being regenerated -- 2.13.6