From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6592507174472122368 X-Received: by 2002:a50:89ca:: with SMTP id h10-v6mr3324001edh.11.1534945307327; Wed, 22 Aug 2018 06:41:47 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:ed0c:: with SMTP id j12-v6ls809999eds.4.gmail; Wed, 22 Aug 2018 06:41:46 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxfZ+H3nt90CoqVC8noEHUcwvHcMz4C0Qv3bg3SUs3Wvyq47YnbG1XtlP6OM2ljLRU1GvX9 X-Received: by 2002:a50:89ca:: with SMTP id h10-v6mr3323994edh.11.1534945306930; Wed, 22 Aug 2018 06:41:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534945306; cv=none; d=google.com; s=arc-20160816; b=K/gQmmRiUuYV+CKcAmRdIb6nnJfybTIK9ICLaMRH4v/zVsxy9VvIr9qNQDNfIpOyhD q+d8XOO9/UzsDk1wVpnjRmhnkr/drSFbP+3zoqEI40whf7v/h0hffFAwJbAobZ5gCCfx UfM3f+UHXT8UYibYNMogAJhDSe4tttij60Uv5NCGkNIjgPoA6/jBcrKCodnyCA7xscLh LkPDkuX3DFsN+d6MTDG232vMjxfE8X2lAixhYU1yp6f6lzTQb/bqJpruuG1XjTX2mrF7 Cqn+s4Avw1s87bVjxRfrS7idLokW3/HLAJuGIQoyLSIIPPg2yrz+nE16nFGXTdG0jPyJ SDdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=uf87a7vZtT/UKgGz3hwwxY4LgVyJf6/1zRb449bZzN4=; b=cDX+rS/pKBg9YqY8V4Xr4ahQ3oKg6iNP/EI+PwDUGbh8wQWIZkNmaMwKa0jhhJRzq2 IKiNWEd2m3OjQihJuzyzrcEFrb2HjnByN4YOqu9xhkZSUSk4pywXB4ldiF2cEniqOb2U 1Y2KwyhB3v6kidA2DKUo1AQlUbvmCnmtSJg2MSav9d23miDtsT1hHdUEYWoIR1TVycjm WRV2O0sHEMlqX5FmHJ/3mO8/J4lMcVPZMTjMpmyGi6jVjVm7ZCdif6JP0AmJrHxcCRAk MVmREHqzq/te7GgwemjGd2rjpw3wTlO2+Zjt3AL2U6PrFjx40K8p6q/Id1jcSqqV9LVV B2Ug== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id b23-v6si82070edn.1.2018.08.22.06.41.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Aug 2018 06:41:46 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w7MDfkvf004620 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Aug 2018 15:41:46 +0200 Received: from md1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id w7MDfkXT001441; Wed, 22 Aug 2018 15:41:46 +0200 Date: Wed, 22 Aug 2018 15:41:45 +0200 From: Henning Schild To: "Hombourger, Cedric" Cc: Jan Kiszka , "isar-users@googlegroups.com" Subject: Re: [PATCH 1/1] dpkg: acquire lock when calling reprepro Message-ID: <20180822154145.07d4a2ca@md1pvb1c.ad001.siemens.net> In-Reply-To: <004d3802fb104470a68eada0c12fae1f@svr-ies-mbx-02.mgc.mentorg.com> References: <1534937676-7628-1-git-send-email-Cedric_Hombourger@mentor.com> <1534937676-7628-2-git-send-email-Cedric_Hombourger@mentor.com> <004d3802fb104470a68eada0c12fae1f@svr-ies-mbx-02.mgc.mentorg.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: l+mFuU+NT68k My first reaction was the same, but in the other direction. I would prefer a bitbake lock over the flock, if possible. Henning Am Wed, 22 Aug 2018 12:48:14 +0000 schrieb "Hombourger, Cedric" : > Thanks Jan for your quick feedback. I have submitted a v2 with > isar.lock removed > > Cedric > > -----Original Message----- > From: Jan Kiszka [mailto:jan.kiszka@siemens.com] > Sent: Wednesday, August 22, 2018 2:20 PM > To: Hombourger, Cedric ; > isar-users@googlegroups.com Subject: Re: [PATCH 1/1] dpkg: acquire > lock when calling reprepro > > On 2018-08-22 13:34, Cedric Hombourger wrote: > > As bitbake performs multi-threaded builds, packages may be built > > while others get deployed. A lock was already acquired within > > dpkg_runbuild but accesses to the isar-apt database were not > > protected from do_deploy_deb. Move the lock from the buildchroot to > > the isar-apt repository and acquire the (same) lock from > > dpkg_runbuild, repo_clean and do_deploy_deb. > > I think I've seen this issue once but forgot to dig into the details. > It looks consistent. > > > > > Signed-off-by: Cedric Hombourger > > --- > > meta/classes/dpkg-base.bbclass | 8 ++++++-- > > meta/recipes-devtools/buildchroot/files/build.sh | 2 +- > > meta/recipes-kernel/linux/files/build-kernel.sh | 2 +- > > 3 files changed, 8 insertions(+), 4 deletions(-) > > > > diff --git a/meta/classes/dpkg-base.bbclass > > b/meta/classes/dpkg-base.bbclass index a41df2c..7137968 100644 > > --- a/meta/classes/dpkg-base.bbclass > > +++ b/meta/classes/dpkg-base.bbclass > > @@ -71,6 +71,8 @@ do_build() { > > mount -t proc none ${BUILDCHROOT_DIR}/proc > > fi' > > > > + # Create database lock file here so we own it (not root) and > > perform actual build > > + touch ${DEPLOY_DIR_APT}/${DISTRO}/db.lock > > dpkg_runbuild > > > > sudo umount ${BUILDROOT} 2>/dev/null || true @@ -82,7 +84,8 > > @@ CLEANFUNCS += "repo_clean" > > repo_clean() { > > PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/') > > if [ -n "${PACKAGES}" ]; then > > - reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > > + flock ${DEPLOY_DIR_APT}/${DISTRO}/db.lock \ > > + reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > > --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \ > > -C main -A ${DISTRO_ARCH} \ > > remove ${DEBDISTRONAME} \ @@ -93,7 +96,8 @@ > > repo_clean() { > > # Install package to Isar-apt > > do_deploy_deb() { > > repo_clean > > - reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > > + flock ${DEPLOY_DIR_APT}/${DISTRO}/db.lock \ > > + reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > > --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \ > > -C main \ > > includedeb ${DEBDISTRONAME} \ > > There is also > > do_deploy_deb[lockfiles] = "${DEPLOY_DIR_APT}/isar.lock" > > in that file. I suppose that can now be dropped, can't it? > > > diff --git a/meta/recipes-devtools/buildchroot/files/build.sh > > b/meta/recipes-devtools/buildchroot/files/build.sh > > index e2dabab..e8a9c17 100644 > > --- a/meta/recipes-devtools/buildchroot/files/build.sh > > +++ b/meta/recipes-devtools/buildchroot/files/build.sh > > @@ -50,7 +50,7 @@ install_cmd="apt-get -o > > Debug::pkgProblemResolver=yes --no-install-recommends -y > > > > # Install all build deps > > mk-build-deps $set_arch -t "${install_cmd}" -i -r > > debian/control -) 42>/dpkg.lock > > +) 42>/isar-apt/db.lock > > > > # If autotools files have been created, update their timestamp to > > # prevent them from being regenerated diff --git > > a/meta/recipes-kernel/linux/files/build-kernel.sh > > b/meta/recipes-kernel/linux/files/build-kernel.sh > > index 8b58e93..2ff5a1e 100644 > > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > > +++ b/meta/recipes-kernel/linux/files/build-kernel.sh > > @@ -34,7 +34,7 @@ REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > > REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > > > # Lock-protected because apt and dpkg do not wait in case of > > contention -flock /dpkg.lock \ > > +flock /isar-apt/db.lock \ > > apt-get install -y -o Debug::pkgProblemResolver=yes \ > > --no-install-recommends ${KBUILD_DEPENDS} > > > > > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate > Competence Center Embedded Linux >