From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6665957031077216256 X-Received: by 2002:adf:f08c:: with SMTP id n12mr1039949wro.7.1552063089844; Fri, 08 Mar 2019 08:38:09 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a7b:cd99:: with SMTP id y25ls125251wmj.2.gmail; Fri, 08 Mar 2019 08:38:09 -0800 (PST) X-Google-Smtp-Source: APXvYqx/AI+EB9pOiYRSN3ECDcXMADjK414h7ZuDxE3iE2CwEoxhMtssemGRK8s5B90EuiKxZPCY X-Received: by 2002:a7b:c18d:: with SMTP id y13mr1585519wmi.2.1552063089279; Fri, 08 Mar 2019 08:38:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1552063089; cv=none; d=google.com; s=arc-20160816; b=gnNk/0PO61/FahKxUGNM9048RRp5+hn3LfvWzBU+ND7LWyUyqvBTKjYpA49E+bYfpl hhqpyqlV35PoCbzvSvbfinW009le3y/ScCcTvbh2WBKXgPo0E5Zyh5VEo0S2o4DJBpYi h2Ti4itteXqydd0XDXEOaNHYLrt6CzZSgTqsOzKlhww7l/EC1gMKq0EoyL3YX7X+HAdY Qn5SUzDeyaxnbfbfAnKpuTMV4jNmuQpwzu6K+vAIBcPgErzf+95EJfYeAiYIDms37QEN 8Bre037oXJqbNIn8zJOGlu6/pk01YsGdytseU7qYC25/vNZrXNaHNYJYJASv5fsfbyX3 090g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=yua37GZPz1mOzli7QuMFooBdz8ptPqIUr7xQ1Y2ZEWI=; b=XaH/u4ym5vDozc1VznaHHdSn3BtjO62wmM6tXYp5/iH0fMeWKCpTanPpPFwuE++X4n p9gBzydQoMJXxtybpUcxoWJQ0gEagWnS1OYypB9uPvH17Ec7RZVeOhREPGMP8ZyW1M8Q NisohMaHcV2fkKgLIYHAki6zDSmMfni8tiWI2q4YD6bOi0vWKZXvkeLylMM4NIOHl9qy V8oR39jmZP3ajB372ogb+iSWd3QXXbgNNgDNs/s77skok2+eIscU/d7lj/gatsJagEwn hGaYzRE/j77VT7SEC8Xr1Fk7CovzoTt9c3YGJlmW1bHb5OpTqxx/kC6Z9cG7KUINxQTf 7sWw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id b8si414255wru.1.2019.03.08.08.38.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Mar 2019 08:38:09 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id x28Gc83W009038 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 8 Mar 2019 17:38:08 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x28Gc8v4001416; Fri, 8 Mar 2019 17:38:08 +0100 Subject: Re: [PATCH v2] dpkg: acquire a read (shared) locks while building packages To: Cedric Hombourger , isar-users@googlegroups.com References: <97148d8a-edea-f4c3-d35d-f538512b7c38@siemens.com> <1552060145-1317-1-git-send-email-Cedric_Hombourger@mentor.com> From: Jan Kiszka Message-ID: Date: Fri, 8 Mar 2019 17:38:07 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <1552060145-1317-1-git-send-email-Cedric_Hombourger@mentor.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: eLCQvezUi/iy On 08.03.19 16:49, Cedric Hombourger wrote: > As debhelper scripts involved during the actual build process > (dpkg-buildpackage) may access the dpkg database, we need to > make sure it does not get modified in the background by other > build tasks for other packages (such as install_build_deps). > An example of a reader is dh_shlibdeps/dpkg-shlibdeps: it > needs to find packages providing the shared libraries the > package depends on to substitute shlibs variables found in its > control file with package dependencies. Acquire the isar-apt > lock as a shared lock to block writers. > > Special care may be needed in other classes/users of the > dpkg_runbuild API. Due to some limitations with bitbake (to be > discussed with the maintainers), we could not grab the lock > from dpkg-base. A statement such as "flock > dpkg_run_build" would fail as flock can't exec() dpkg_run_build > (since it is a shell function, not an executable). The popular > shell construct "( flock -n 9; dpkg_run_build ) 9> " > cannot be parsed by bitbake. And finally the lockfiles task > attribute (implemented in bitbake) only handles exclusive locks. > > This commit is only addressing dpkg.bbclass. dpkg-raw.bbclass > is left intact as the generated control file does require ( Actually, dpkg-raw just uses dpkg these days, so no work to be done anyway. ) > dh_shlibdeps. linux-custom.inc was also left out as the Debian > packaging scripts only use low-level packaging tools and do not > use debhelper scripts during packaging (at this time). In > agreement with the Isar project maintainers, we will not grab > the Isar lock from linux-custom.inc to let kernel builds run > in parallel with do_install_deps() and do_deploy_deb() that > may be executed by other bitbake threads. > > Signed-off-by: Cedric Hombourger > --- > meta/classes/dpkg.bbclass | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > index 24b9fe3..f4e3d7a 100644 > --- a/meta/classes/dpkg.bbclass > +++ b/meta/classes/dpkg.bbclass > @@ -19,5 +19,6 @@ do_install_builddeps[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > # Build package from sources using build script > dpkg_runbuild() { > E="${@ bb.utils.export_proxies(d)}" > + flock -s "${REPO_ISAR_DIR}/isar.lock" \ > sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} /isar/build.sh ${PP}/${PPS} ${DISTRO_ARCH} > } > Looks good to me. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux