public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
To: <isar-users@googlegroups.com>
Cc: Cedric Hombourger <Cedric_Hombourger@mentor.com>
Subject: [PATCH 1/1] dpkg: acquire lock when calling reprepro
Date: Wed, 22 Aug 2018 13:34:36 +0200	[thread overview]
Message-ID: <1534937676-7628-2-git-send-email-Cedric_Hombourger@mentor.com> (raw)
In-Reply-To: <1534937676-7628-1-git-send-email-Cedric_Hombourger@mentor.com>

As bitbake performs multi-threaded builds, packages may be built
while others get deployed. A lock was already acquired within
dpkg_runbuild but accesses to the isar-apt database were not
protected from do_deploy_deb. Move the lock from the buildchroot
to the isar-apt repository and acquire the (same) lock from
dpkg_runbuild, repo_clean and do_deploy_deb.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/dpkg-base.bbclass                   | 8 ++++++--
 meta/recipes-devtools/buildchroot/files/build.sh | 2 +-
 meta/recipes-kernel/linux/files/build-kernel.sh  | 2 +-
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index a41df2c..7137968 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -71,6 +71,8 @@ do_build() {
             mount -t proc none ${BUILDCHROOT_DIR}/proc
         fi'
 
+    # Create database lock file here so we own it (not root) and perform actual build
+    touch ${DEPLOY_DIR_APT}/${DISTRO}/db.lock
     dpkg_runbuild
 
     sudo umount ${BUILDROOT} 2>/dev/null || true
@@ -82,7 +84,8 @@ CLEANFUNCS += "repo_clean"
 repo_clean() {
     PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/')
     if [ -n "${PACKAGES}" ]; then
-        reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
+        flock ${DEPLOY_DIR_APT}/${DISTRO}/db.lock \
+            reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
                  --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \
                  -C main -A ${DISTRO_ARCH} \
                  remove ${DEBDISTRONAME} \
@@ -93,7 +96,8 @@ repo_clean() {
 # Install package to Isar-apt
 do_deploy_deb() {
     repo_clean
-    reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
+    flock ${DEPLOY_DIR_APT}/${DISTRO}/db.lock \
+        reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
              --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \
              -C main \
              includedeb ${DEBDISTRONAME} \
diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh
index e2dabab..e8a9c17 100644
--- a/meta/recipes-devtools/buildchroot/files/build.sh
+++ b/meta/recipes-devtools/buildchroot/files/build.sh
@@ -50,7 +50,7 @@ install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y
 
     # Install all build deps
     mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control
-) 42>/dpkg.lock
+) 42>/isar-apt/db.lock
 
 # If autotools files have been created, update their timestamp to
 # prevent them from being regenerated
diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 8b58e93..2ff5a1e 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -34,7 +34,7 @@ REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image"
 REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers"
 
 # Lock-protected because apt and dpkg do not wait in case of contention
-flock /dpkg.lock \
+flock /isar-apt/db.lock \
     apt-get install -y -o Debug::pkgProblemResolver=yes \
         --no-install-recommends ${KBUILD_DEPENDS}
 
-- 
2.11.0


  reply	other threads:[~2018-08-22 11:34 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-22 11:34 [PATCH 0/1] acquire database " Cedric Hombourger
2018-08-22 11:34 ` Cedric Hombourger [this message]
2018-08-22 12:20   ` [PATCH 1/1] dpkg: acquire " Jan Kiszka
2018-08-22 12:48     ` Hombourger, Cedric
2018-08-22 13:41       ` Henning Schild
2018-08-22 13:47         ` Hombourger, Cedric
2018-08-22 13:48         ` Jan Kiszka
2018-08-26  6:21         ` [PATCH v3 1/3] dpkg: move installation of dependencies into dpkg_prepare Cedric Hombourger
2018-08-26  6:21           ` [PATCH v3 2/3] linux: leverage dpkg_prepare to install build dependencies Cedric Hombourger
2018-08-26  6:21           ` [PATCH v3 3/3] dpkg+linux: use Isar's lock via do_prepare[lockfiles] instead of flock Cedric Hombourger
2018-08-28 12:12           ` [PATCH v3 1/3] dpkg: move installation of dependencies into dpkg_prepare Maxim Yu. Osipov
2018-08-31  8:10           ` Jan Kiszka
2018-08-31  8:15             ` chombourger
2018-08-31  8:21               ` Jan Kiszka
2018-08-31  8:28           ` Jan Kiszka
2018-08-31  8:33             ` chombourger
2018-08-31  8:41               ` Jan Kiszka
2018-08-31  9:05                 ` Jan Kiszka
2018-09-01  6:58                   ` Jan Kiszka
2018-09-01  8:44                     ` Cedric Hombourger
2018-09-01  9:31                       ` Jan Kiszka
2018-08-22 11:58 ` [PATCH 0/1] acquire database lock when calling reprepro Alexander Smirnov
2018-08-22 12:07   ` Hombourger, Cedric
2018-08-22 12:47 ` [PATCH v2] dpkg: acquire " Cedric Hombourger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1534937676-7628-2-git-send-email-Cedric_Hombourger@mentor.com \
    --to=cedric_hombourger@mentor.com \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox