From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6665957031077216256 X-Received: by 2002:a2e:8ec9:: with SMTP id e9mr68651ljl.6.1552625251524; Thu, 14 Mar 2019 21:47:31 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:4d0:: with SMTP id a77ls999298ljf.1.gmail; Thu, 14 Mar 2019 21:47:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqzHVn2PKO47QoJRFjxREFIc4UPukEfZQ3ORSfRnDlQNACJm0aot8E5F4siQts0TBBOk3EKx X-Received: by 2002:a2e:8e85:: with SMTP id z5mr73925ljk.16.1552625251049; Thu, 14 Mar 2019 21:47:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552625251; cv=none; d=google.com; s=arc-20160816; b=eV06VEea+1/zLIUX+TStPH/sprC/LKAB1F5DgilFtH4dsC5cE6xJRyz0y8Kf1hHjlN MKdEmGM4kusBtxnjqkdB+Np5rQqvP4n0jGucQalFryEkBob92fHqFz+jaHJUXO5HXdc2 bgYN9FZBrWazSTSPkGhPTMFnBzAK68MkwnAXc0IsIVCIDN9gAXZNT6oDW7cj+GNscrt3 gNURD1Z0khNfj0kGtlKEt5k4pfzz5u/yWF5Wr3EC8Ec0wzIeoHPRDHpGMQmy1HdquN9u O7d2BlPlHe6HGbWQhHjZCRSzmDNf6jK7EAOgUTYr6tOQEJ/JdgbIvl6syd3VSJlPFMM6 Vtzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:to:subject; bh=6VyHNiimgbWNlvEGfyUCadYKbVsXJloKgybTbfDg+T4=; b=l6d+wmLMbnmWMK5EpYHeQ0Tf3q1Z+5U4up/xe35tYXvovd0UMiwpKw/mCI7/7u2f2x 2LTfe+Abea3ig3clHGe1tsTr3MrCFkuoBkyyj6zVS2eRvJ0OFWdbfJOkrEyWi2bx4SLZ Jz6GkMHwXlRf5p7gUVXR9Pd0nh3boZzjLPhw9/TO3clylsW+5hJGQImqVE3VqD6CmLzI 36EE1QoXd/jscCnjkSzixhhWrlj2ZVsYnPTPDJRwR0E36kcyxWuEyMChkvNyQYtPIO3E yGMvkFA0ChtKb8/T2dYDfPCWG1NAUiTnR/FsGcnGGusL6amg6Ap3YWDwXHjg4u1+VeJu saOQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id k13si24600lja.5.2019.03.14.21.47.30 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Mar 2019 21:47:30 -0700 (PDT) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from [192.168.50.163] (d51A48A80.access.telenet.be [81.164.138.128]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x2F4lT6a029091 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 15 Mar 2019 05:47:29 +0100 Subject: Re: [PATCH v2] dpkg: acquire a read (shared) locks while building packages To: Cedric Hombourger , isar-users@googlegroups.com References: <97148d8a-edea-f4c3-d35d-f538512b7c38@siemens.com> <1552060145-1317-1-git-send-email-Cedric_Hombourger@mentor.com> From: "Maxim Yu. Osipov" Organization: ilbers GmbH Message-ID: Date: Fri, 15 Mar 2019 05:47:28 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <1552060145-1317-1-git-send-email-Cedric_Hombourger@mentor.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: OB9E8CrXpB+X On 3/8/19 4:49 PM, Cedric Hombourger wrote: > As debhelper scripts involved during the actual build process > (dpkg-buildpackage) may access the dpkg database, we need to > make sure it does not get modified in the background by other > build tasks for other packages (such as install_build_deps). > An example of a reader is dh_shlibdeps/dpkg-shlibdeps: it > needs to find packages providing the shared libraries the > package depends on to substitute shlibs variables found in its > control file with package dependencies. Acquire the isar-apt > lock as a shared lock to block writers. > > Special care may be needed in other classes/users of the > dpkg_runbuild API. Due to some limitations with bitbake (to be > discussed with the maintainers), we could not grab the lock > from dpkg-base. A statement such as "flock > dpkg_run_build" would fail as flock can't exec() dpkg_run_build > (since it is a shell function, not an executable). The popular > shell construct "( flock -n 9; dpkg_run_build ) 9> " > cannot be parsed by bitbake. And finally the lockfiles task > attribute (implemented in bitbake) only handles exclusive locks. > > This commit is only addressing dpkg.bbclass. dpkg-raw.bbclass > is left intact as the generated control file does require > dh_shlibdeps. linux-custom.inc was also left out as the Debian > packaging scripts only use low-level packaging tools and do not > use debhelper scripts during packaging (at this time). In > agreement with the Isar project maintainers, we will not grab > the Isar lock from linux-custom.inc to let kernel builds run > in parallel with do_install_deps() and do_deploy_deb() that > may be executed by other bitbake threads. Applied to the 'next'. Thanks, Maxim. > Signed-off-by: Cedric Hombourger > --- > meta/classes/dpkg.bbclass | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > index 24b9fe3..f4e3d7a 100644 > --- a/meta/classes/dpkg.bbclass > +++ b/meta/classes/dpkg.bbclass > @@ -19,5 +19,6 @@ do_install_builddeps[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > # Build package from sources using build script > dpkg_runbuild() { > E="${@ bb.utils.export_proxies(d)}" > + flock -s "${REPO_ISAR_DIR}/isar.lock" \ > sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} /isar/build.sh ${PP}/${PPS} ${DISTRO_ARCH} > } > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov