From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6521316140522668032 X-Received: by 10.80.184.23 with SMTP id j23mr2438890ede.3.1518368123861; Sun, 11 Feb 2018 08:55:23 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.185.67 with SMTP id m61ls5462072ede.4.gmail; Sun, 11 Feb 2018 08:55:23 -0800 (PST) X-Google-Smtp-Source: AH8x225Rdbs5zPp6QpOd24dftIYdWveNDzESY+/npf4JDsqw7TXb5SR5yoreF9TLvAVHCabzxWTh X-Received: by 10.80.184.23 with SMTP id j23mr2438888ede.3.1518368123372; Sun, 11 Feb 2018 08:55:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518368123; cv=none; d=google.com; s=arc-20160816; b=sWkTWJqw5kNrbAsuBokz37Qmr4OB4kcC3qg8H97oLnHOKUFUxD2aOHQJtRuzlZkHof CsBjyO3uUAnzW4P+hSmmZOfoEZWdfiKGB2au1gw10jbgMOEDOlD/1bDbq38DFmcdwHwY vrfP17/qapAcz3IF3+y0olWlCCrsxM7qsTxnx6hMVurTW1xDxeTTfLkeSaMPphRmWXRU +awqFcshUd4fyp4dj9E+GVxt88cqugyfH3fJ7XuGEVwOsLqsmiR7r86qgr/2IZkxM7tL pZh7wFFIuZ16r15KFw4JZBMJMjEPgzmHtyFgFlvxA4GJZSigemUzUAbjYNdgEhCcr7dK RQLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=jSv2hxqDHDDbtSMd66XMevIpzHQ7mrn+MRwCql0SE+I=; b=09B/4FbunocYLHBqGHnDoCsxdzkmQ5r1gBTZoT7AomR02tdvIpUslqHa4fb/dsGW9m 8l1i6+ozW34RWh4Sl1xYsQP3NSBwUp/26VpneY0vBLziBhhLMJCgEfi05PWVVMGiJiTj TU68sBWW6om0gFeFUFyVOhMOhkyvivPQNQ5ajBvVZi2s5eU6UXMhxCn6Fr/B0gzsaPx2 P3t9h5roU8BpxN4cE66IFbgYb5qE//LA/lfh6heHKi0dYxiAN1Rz6ZEX4DLlRg2Rhprt LU5eK+TmS6yFHhJKV63Izlk9ZmtXYNIw4j/MCjVfZpVe2iFtmIKFWwKdKynbp6JWL4SY vNJw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id z17si478352edz.1.2018.02.11.08.55.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Feb 2018 08:55:23 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w1BGtJE4029305 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sun, 11 Feb 2018 17:55:21 +0100 Subject: Re: Build can sporadically fail due to dpkg contention in buildchroot To: Jan Kiszka , isar-users References: <79bd216e-53ca-5485-4f6c-66050d08ed5f@siemens.com> From: Alexander Smirnov Message-ID: <4d2b9322-54ed-ddff-739e-d0a3d5c6cc7b@ilbers.de> Date: Sun, 11 Feb 2018 19:55:14 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <79bd216e-53ca-5485-4f6c-66050d08ed5f@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 3C14Bf4i4pn7 Hi, On 02/11/2018 06:17 PM, Jan Kiszka wrote: > Hi all, > > I got this failure of example-hello:do_build already twice while doing > rebuild tests with my kernel series (ie. with more independent > buildchroot users): > > DEBUG: Executing shell function do_build > Get:1 file:/isar-apt isar InRelease > Ign:1 file:/isar-apt isar InRelease > Get:2 file:/isar-apt isar Release [2,864 B] > Get:2 file:/isar-apt isar Release [2,864 B] > Get:3 file:/isar-apt isar Release.gpg > Ign:3 file:/isar-apt isar Release.gpg > Get:4 file:/isar-apt isar/main amd64 Packages [1,135 B] > Reading package lists... > W: The repository 'file:/isar-apt isar Release' is not signed. > hostname: No address associated with hostname > dh_testdir > dh_testroot > dh_prep > dh_testdir > dh_testroot > dh_install > dh_install: Compatibility levels before 9 are deprecated (level 7 in use) > dh_installdocs > dh_installdocs: Compatibility levels before 9 are deprecated (level 7 in use) > dh_installchangelogs > dh_compress > dh_fixperms > dh_installdeb > dh_installdeb: Compatibility levels before 9 are deprecated (level 7 in use) > dh_gencontrol > dh_md5sums > dh_builddeb > dpkg-deb: building package 'hello-build-deps' in '../hello-build-deps_0.2_all.deb'. > Good catch! > The package has been created. > Attention, the package has been created in the current directory, > not in ".." as indicated by the message above! > dpkg: error: dpkg status database is locked by another process > mk-build-deps: dpkg --unpack failed > [...] > > So we have a concurrency problem when building over the same dpkg > database. Looks like we need to synchronize (lock-protect) the access to > it, which also means pulling out the dependency installation from the > regular build step. Is that feasible at all? Any alternatives (besides > retrying such builds...)? In general we could do this easily: 1. Split the content of build.sh into two functions, for example: - install_build_deps - build_package 2. Spit the bitbake do_build() into two tasks: do_install_build_deps() { ... build.sh install_build_deps ... } addtask install_build_deps before do_build after do_unpack do_build_package() { ... build.sh build_package ... } 3. Using bitbake synchronization primitives, protect the fist task from parallel execution. If you are OK with this, I could do this tomorrow. Alex