From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6592507174472122368 X-Received: by 2002:a19:db5b:: with SMTP id s88-v6mr2684748lfg.30.1534940408712; Wed, 22 Aug 2018 05:20:08 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:1287:: with SMTP id 7-v6ls70266lfs.4.gmail; Wed, 22 Aug 2018 05:20:08 -0700 (PDT) X-Google-Smtp-Source: AA+uWPzcyWK/c3MXjoUCAxIAK9fXMblVHdGHyPE49JJw2Fd90mEr4JKwzKjJiKe2zdRo7TtjSYie X-Received: by 2002:a19:915c:: with SMTP id y28-v6mr777320lfj.37.1534940408063; Wed, 22 Aug 2018 05:20:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534940408; cv=none; d=google.com; s=arc-20160816; b=IpRL+3+skD8syuil2HlBtJyHDVnJihcNZvl2mL3wnCq9pY7UspqMGOTQZ078fYUi1o bKBmX8A3z3ThToR1ToWnLDX5bMz9FSzenx72dKDnicYYrO0bUNODAz4OcaMwVe+kYxKF 2ePdZ5+5TRqQUenpy35YVF/CxmH1dyqryCTUX4NQtc+9h10RIYxUNosMzgumAD0t1IV7 JpwAGTAt9OpOp2Li2Fj8aiPLdg11L82gQpHfv3W4oM6+ghptqsbl7P9YSicstB/Km++n gRmchk4IdDGsfMzfWDPZCDI5Aiuecq7tI5K2ra9gOEhU4Fx/SPLMDewZv7RCq4UTdEDx fRiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=rCIV4D6wtwJqQ4h1HY1OcTi3U+k92C/17GxuyQ3NgCs=; b=ymeJNXjU9h22pPcZY8Q5gyRZPHoVFXPeZzm9OfMonJZTyvWxOmGgOLzRMDNh7K8Wdy Y8o3hy4PzxvLvE6n/e+b7GCe9Y89hVC18KiNy58IsdgtZxOCPe9ke7530VpuBhvnSvv+ SseFo8+LqGZlgYeUQ5CHLhSKjUWxbmyRTBLpmkg9+wb2AGrU2JqHJNnFht7OuVkvEUCT 0VqDXXq3ZbRw4wxt1yt25RfRz25wf82yrhI+F7hY7gWK4K25KuIpjH/hbQaWlRcyq9m2 8R5XCWIzQVigJgIefeIxc7PymKnT0PdbO+XOv6bQwraB+fROVirI0LTCNQbjB1e6FRfU vb3g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id w85-v6si47288lfk.3.2018.08.22.05.20.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Aug 2018 05:20:08 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w7MCK7cn027379 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Aug 2018 14:20:07 +0200 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id w7MCK6Bo015697; Wed, 22 Aug 2018 14:20:07 +0200 Subject: Re: [PATCH 1/1] dpkg: acquire lock when calling reprepro To: Cedric Hombourger , isar-users@googlegroups.com References: <1534937676-7628-1-git-send-email-Cedric_Hombourger@mentor.com> <1534937676-7628-2-git-send-email-Cedric_Hombourger@mentor.com> From: Jan Kiszka Message-ID: Date: Wed, 22 Aug 2018 14:20:06 +0200 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <1534937676-7628-2-git-send-email-Cedric_Hombourger@mentor.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: xWcJpySa2fSV On 2018-08-22 13:34, Cedric Hombourger wrote: > As bitbake performs multi-threaded builds, packages may be built > while others get deployed. A lock was already acquired within > dpkg_runbuild but accesses to the isar-apt database were not > protected from do_deploy_deb. Move the lock from the buildchroot > to the isar-apt repository and acquire the (same) lock from > dpkg_runbuild, repo_clean and do_deploy_deb. I think I've seen this issue once but forgot to dig into the details. It looks consistent. > > Signed-off-by: Cedric Hombourger > --- > meta/classes/dpkg-base.bbclass | 8 ++++++-- > meta/recipes-devtools/buildchroot/files/build.sh | 2 +- > meta/recipes-kernel/linux/files/build-kernel.sh | 2 +- > 3 files changed, 8 insertions(+), 4 deletions(-) > > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index a41df2c..7137968 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -71,6 +71,8 @@ do_build() { > mount -t proc none ${BUILDCHROOT_DIR}/proc > fi' > > + # Create database lock file here so we own it (not root) and perform actual build > + touch ${DEPLOY_DIR_APT}/${DISTRO}/db.lock > dpkg_runbuild > > sudo umount ${BUILDROOT} 2>/dev/null || true > @@ -82,7 +84,8 @@ CLEANFUNCS += "repo_clean" > repo_clean() { > PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/') > if [ -n "${PACKAGES}" ]; then > - reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > + flock ${DEPLOY_DIR_APT}/${DISTRO}/db.lock \ > + reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \ > -C main -A ${DISTRO_ARCH} \ > remove ${DEBDISTRONAME} \ > @@ -93,7 +96,8 @@ repo_clean() { > # Install package to Isar-apt > do_deploy_deb() { > repo_clean > - reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > + flock ${DEPLOY_DIR_APT}/${DISTRO}/db.lock \ > + reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ > --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \ > -C main \ > includedeb ${DEBDISTRONAME} \ There is also do_deploy_deb[lockfiles] = "${DEPLOY_DIR_APT}/isar.lock" in that file. I suppose that can now be dropped, can't it? > diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh > index e2dabab..e8a9c17 100644 > --- a/meta/recipes-devtools/buildchroot/files/build.sh > +++ b/meta/recipes-devtools/buildchroot/files/build.sh > @@ -50,7 +50,7 @@ install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y > > # Install all build deps > mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control > -) 42>/dpkg.lock > +) 42>/isar-apt/db.lock > > # If autotools files have been created, update their timestamp to > # prevent them from being regenerated > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh > index 8b58e93..2ff5a1e 100644 > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > +++ b/meta/recipes-kernel/linux/files/build-kernel.sh > @@ -34,7 +34,7 @@ REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > # Lock-protected because apt and dpkg do not wait in case of contention > -flock /dpkg.lock \ > +flock /isar-apt/db.lock \ > apt-get install -y -o Debug::pkgProblemResolver=yes \ > --no-install-recommends ${KBUILD_DEPENDS} > > Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux