public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Jan Kiszka <jan.kiszka@siemens.com>
To: isar-users <isar-users@googlegroups.com>,
	Alexander Smirnov <asmirnov@ilbers.de>
Cc: "Larson, Chris" <Chris_Larson@mentor.com>
Subject: [PATCH] buildchroot: Lock-protect apt/dpkg operations
Date: Wed, 14 Feb 2018 20:42:08 +0100	[thread overview]
Message-ID: <92b096db-6db3-401c-0bba-09a31cff2376@siemens.com> (raw)

From: Jan Kiszka <jan.kiszka@siemens.com>

When running multiple build targets in parallel, they will compete for
access on the package database while the related commands do not wait
but fail immediately.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

Fixes parallel builds (CI setup) for me.

I just recalled that Chris was reporting the same issue a long time ago,
but his suggestion to lock down whole do_build was not that well
received...

 meta/recipes-devtools/buildchroot/files/build.sh | 32 +++++++++++++++---------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh
index 023a6d6..048f76f 100644
--- a/meta/recipes-devtools/buildchroot/files/build.sh
+++ b/meta/recipes-devtools/buildchroot/files/build.sh
@@ -2,17 +2,9 @@
 #
 # This software is a part of ISAR.
 # Copyright (C) 2015-2017 ilbers GmbH
+# Copyright (c) 2018 Siemens AG
 
-# Make sure that we have latest isar-apt content.
-# Options meaning:
-#   Dir::Etc::sourcelist - specifies which source to be used
-#   Dir::Etc::sourceparts - disables looking for the other sources
-#   APT::Get::List-Cleanup - do not erase obsolete packages list for
-#                            upstream in '/var/lib/apt/lists'
-apt-get update \
-    -o Dir::Etc::sourcelist="sources.list.d/multistrap-isar-apt.list" \
-    -o Dir::Etc::sourceparts="-" \
-    -o APT::Get::List-Cleanup="0"
+set -e
 
 # Go to build directory
 cd $1
@@ -26,8 +18,24 @@ install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y
 # Allow unauthenticated feeds
 install_cmd="${install_cmd} --allow-unauthenticated"
 
-# Install all build deps
-mk-build-deps -t "${install_cmd}" -i -r debian/control
+(
+    # Lock-protected because apt and dpkg do not wait in case of contention
+    flock 42 || exit 1
+
+    # Make sure that we have latest isar-apt content.
+    # Options meaning:
+    #   Dir::Etc::sourcelist - specifies which source to be used
+    #   Dir::Etc::sourceparts - disables looking for the other sources
+    #   APT::Get::List-Cleanup - do not erase obsolete packages list for
+    #                            upstream in '/var/lib/apt/lists'
+    apt-get update \
+        -o Dir::Etc::sourcelist="sources.list.d/multistrap-isar-apt.list" \
+        -o Dir::Etc::sourceparts="-" \
+        -o APT::Get::List-Cleanup="0"
+
+    # Install all build deps
+    mk-build-deps -t "${install_cmd}" -i -r debian/control
+) 42>/dpkg.lock
 
 # If autotools files have been created, update their timestamp to
 # prevent them from being regenerated
-- 
2.13.6

             reply	other threads:[~2018-02-14 19:42 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-14 19:42 Jan Kiszka [this message]
2018-02-15  8:16 ` Alexander Smirnov
2018-02-15  8:46   ` Jan Kiszka
2018-02-15  9:39 ` Alexander Smirnov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=92b096db-6db3-401c-0bba-09a31cff2376@siemens.com \
    --to=jan.kiszka@siemens.com \
    --cc=Chris_Larson@mentor.com \
    --cc=asmirnov@ilbers.de \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox