From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6509455202632859648 X-Received: by 10.36.104.131 with SMTP id v125mr1117843itb.8.1515684075641; Thu, 11 Jan 2018 07:21:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.107.102.14 with SMTP id a14ls298447ioc.14.gmail; Thu, 11 Jan 2018 07:21:15 -0800 (PST) X-Google-Smtp-Source: ACJfBosFW4K36oOzsAwap15GG9NZxf29OGkzvB/QDNYXdTIO7x5KVx3iFiwpjjV3du7JhiajSuGT X-Received: by 10.107.55.66 with SMTP id e63mr15708213ioa.139.1515684075381; Thu, 11 Jan 2018 07:21:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515684075; cv=none; d=google.com; s=arc-20160816; b=e6rzubSJQQP9uPcg3pAS+3XTbSslIa55pthZk41dVqoDFCB9OX04BUTh9YhG2+djEs lfV2TSB60DxzNPd3g11lYLr7bG8aceTNTwKZAUNPwuWqc5Yl4qPjG6+JupP2Gz0pYsAI ZI0upfEg+QACuoGhvtD9BxyD5/jy8YPuCgg//biMTMLDUR/640iKQtiMuE0csL7OJwUn ret0cCfqjp8lgoe2+OA35LkKv9goYAI1mohbtU7ex7M+CydFBrSwrHauUmZbb8giUdDg yBzYPEU2GLo0JYLRm/KBLU8DQz1OBkBat3jf5i8n5Y57InFd9q7XyFreYObEOrnkAfNo Fozg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:dkim-signature:arc-authentication-results; bh=gaiBFj8qAW/I4n7NOnbzCeYZhKfVdU82SKYeCiiUatk=; b=ISAzv5TWzZT8keGIZV4bruT2UBHnKA7jlQjqtKvbJ6qON7XNck886afT0YfPDBCQeU 8wnAD8ddvpIYOM5mBdfGEJ/zZNcZVn99lL8bPIraYA0Kuirk8EHDEHBpTydlxBuQOqTL ivpywCYpeozQR8y1QATc+9HbhAynJ3yLQI1vTFXyEepr41KJ5F6/k6v2ZDTAAgx4fw0A IbX2O7ImPFeKJPA0CKOeORKbPTS9U50Nzsn9rtQB7bhQcMAkV1FVmjfN5dM7Mpl80Nps DEFQGFUHSXW81k8dyJ4N+q4gPqxKHbGUgFTZlEtJ3ENLi42GNVfjsxlVawotStX9Z0qt d7WA== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=aDnJKHQz; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:400c:c05::242 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-vk0-x242.google.com (mail-vk0-x242.google.com. [2607:f8b0:400c:c05::242]) by gmr-mx.google.com with ESMTPS id q25si40921ioh.1.2018.01.11.07.21.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Jan 2018 07:21:15 -0800 (PST) Received-SPF: pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:400c:c05::242 as permitted sender) client-ip=2607:f8b0:400c:c05::242; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=aDnJKHQz; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:400c:c05::242 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: by mail-vk0-x242.google.com with SMTP id w22so1717643vkw.0 for ; Thu, 11 Jan 2018 07:21:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=gaiBFj8qAW/I4n7NOnbzCeYZhKfVdU82SKYeCiiUatk=; b=aDnJKHQzLcL5uyY+Wky5z/546GxQQEwEvKOMImdqEK3bVtQyWcYpXW17yxcl6WhfSL Mujnz/8xsHelZavcz9GbMQOJwd8VYDvizMapVR8SIcrikIa4lbYjsge4JQmKo1pShWg1 aosIOwwJUnKwjmTSwC0fsMVwSw0n2siFlSRGJdsq+bfCN4zXOl2MMf/i0mlLIOajPyCu OPJwUzmTcWa92xx71djp6sZpA0XH1nNXD6zO9Hk9cVM0D4XPqb+zEoWdR+qJ5qpeU0jM tcTZToO5hv1vJ2bdjajE1jahXKNxOUv3XMgcSPwd0V7VnnKu4pHw4ozJ5ezqRU/hLbzq DBjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=gaiBFj8qAW/I4n7NOnbzCeYZhKfVdU82SKYeCiiUatk=; b=nH2Mz5vRvvGOfHzuSRx1i70Jk8vhVWyWVVs0cRsf0lVWal6wdz+NWWdehAy4NXQkDw wJl+TM8iUPIKtPCKz/IamwBKrjBRXQ1yibbzCeW2olJ7VFXBKyjjWkfkHfrZXHGLWElj /Jy3LZq7wih9I17DFS7ul5B96mjV08UZQHvEOuj+LZiun6jmsCuUHCOotWdLqxN42xid 3EayLjc9qFdDWhc4ay6sonetHhq/LKcj+V6NCKByya5Iz85qxTashIdiMGB/XPRxYDX0 SHN1gCtQIfKgtaXcIJSfzy1H4Hb3bIKuGocIijnyg7EB1NFF9cKbrkdXyhAMkUgn9Ajz 5YyQ== X-Gm-Message-State: AKwxytc0JDMTdMxBX/4b3yrqiAdGR4uLK2GJWrCHaExzVFZixoBPIlZJ f7UrLIzBPbjvZTxiqCecYqruFHi9dJ9h/OJKxr0= X-Received: by 10.31.153.76 with SMTP id b73mr19993244vke.53.1515684075007; Thu, 11 Jan 2018 07:21:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.159.55.142 with HTTP; Thu, 11 Jan 2018 07:20:54 -0800 (PST) In-Reply-To: <20180111131425.4d6035c1@mmd1pvb1c.ad001.siemens.net> References: <20180110160409.18273-1-kergoth@gmail.com> <20180111131425.4d6035c1@mmd1pvb1c.ad001.siemens.net> From: Christopher Larson Date: Thu, 11 Jan 2018 08:20:54 -0700 Message-ID: Subject: Re: [PATCH] dpkg.bbclass: add lockfile for do_build To: Henning Schild Cc: Alexander Smirnov , isar-users@googlegroups.com Content-Type: multipart/alternative; boundary="001a1140f616e1f679056281b296" X-TUID: D9WwSQtfxaSL --001a1140f616e1f679056281b296 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jan 11, 2018 at 5:14 AM, Henning Schild wrote: > Am Thu, 11 Jan 2018 13:31:45 +0300 > schrieb Alexander Smirnov : > > > Hello Christopher, > > > > thank you for this catch! > > > > On 01/10/2018 07:04 PM, Christopher Larson wrote: > > > All recipes using this class will run build.sh in the same chroot, > > > and build.sh runs apt-get to install deps, so the build can fail > > > due to dpkg being unable to acquire its lock. Add a bitbake > > > lockfile to ensure that only one do_build task for recipes > > > inheriting this class runs at a time. > > > > Serialization of do_build tasks leads to whole Isar serialization, > > what is significant issue if you have lots of packages to build. So I > > think the "build" and "deps installation" should be split in separate > > tasks and only the second one should be protected by locks. > > The fact that Isar uses multiconfig to build many targets from the same > tree is not intuitive, we discussed this before. So i am not surprised > new contributors get that "wrong". Alex i suggest you split the task > and adopt Christophers changes. > > Christopher, have a look at scripts/ci_build.sh and you will see that > Isar is capable of building multiple images at once in the same tree. > If you are building 10 images you will have 10 buildchroots that have > to be serialized seperately. A feature that probably nobody but the > CI-bot will ever use ;). Fair enough, thanks, I=E2=80=99ll re-test with 1) a lockfile in the buildch= root or 2) having build.sh wait on a lock itself, if possible. --=20 Christopher Larson kergoth at gmail dot com Founder - BitBake, OpenEmbedded, OpenZaurus Senior Software Engineer, Mentor Graphics --001a1140f616e1f679056281b296 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Jan 11, 2018 at 5:14 AM, Henning Schild <<= a href=3D"mailto:henning.schild@siemens.com" target=3D"_blank">henning.schi= ld@siemens.com> wrote:
Am T= hu, 11 Jan 2018 13:31:45 +0300
schrieb Alexander Smirnov <asmirno= v@ilbers.de>:

> Hello Christopher,
>
> thank you for this catch!
>
> On 01/10/2018 07:04 PM, Christopher Larson wrote:
> > All recipes using this class will run build.sh in the same chroot= ,
> > and build.sh runs apt-get to install deps, so the build can fail<= br> > > due to dpkg being unable to acquire its lock. Add a bitbake
> > lockfile to ensure that only one do_build task for recipes
> > inheriting this class runs at a time.
>
> Serialization of do_build tasks leads to whole Isar serializati= on,
> what is significant issue if you have lots of packages to build. So I<= br> > think the "build" and "deps installation" should b= e split in separate
> tasks and only the second one should be protected by locks.

The fact that Isar uses multiconfig to build many targets from the same
tree is not intuitive, we discussed this before. So i am not surprised
new contributors get that "wrong". Alex i suggest you split the t= ask
and adopt Christophers changes.

Christopher, have a look at scripts/ci_build.sh and you will see that
Isar is capable of building multiple images at once in the same tree.
If you are building 10 images you will have 10 buildchroots that have
to be serialized seperately. A feature that probably nobody but the
CI-bot will ever use ;).

Fair enough, thank= s, I=E2=80=99ll re-test with 1) a lockfile in the buildchroot or 2) having = build.sh wait on a lock itself, if possible.
--
Christopher Larson
kergoth at gmail dot com
Found= er - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor = Graphics
--001a1140f616e1f679056281b296--