From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6521574339082452992 X-Received: by 10.25.125.68 with SMTP id y65mr382028lfc.15.1518617189347; Wed, 14 Feb 2018 06:06:29 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.44.194 with SMTP id s185ls1574978lfs.15.gmail; Wed, 14 Feb 2018 06:06:28 -0800 (PST) X-Google-Smtp-Source: AH8x226IHK/elK5oMuaJxFVOOiLjX/+MKnERlaw0U+dDiPl4gJVByFbiQvGmv80QQC4Srd7vdq/+ X-Received: by 10.25.78.9 with SMTP id c9mr374355lfb.24.1518617188652; Wed, 14 Feb 2018 06:06:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518617188; cv=none; d=google.com; s=arc-20160816; b=Km62cb7ElWet3Q5UV7Q6Sil2+tgs9GLJ8DnJ8RJeEw1syNlo1vZ64BGOjUqZ17wUFB S06n/BffL7qD3hOzr572NNCgYmr78dP6QwQ8R6uGUIVOzdw2S3Qefsqfr2Q4C1Fn8YPd KCiz1PQkZByn0jn9hYSg2EItWt0Hq1JLpj4FzsTwRGLnRyQrRSId89KceQOy/k2QopkH FHeZ5G5ysKqX56529MXDfHbJxTlZcvrjcVH4HkHOkdSnls1gXssrvOJMl/QcmXJLdAKy kmSux5fCBpkCxXCQTYBNkFRO8wlSDCu0zzCU2DpfXeyoI7Q1IMTfPbRiHeQ/kJw48c7U jGsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=IUu1mKrkc61cuaaoen4QPniP8j0EZGTrveec9SyZrcs=; b=kfyC4GLvPYIBd9nUwVRrmibWYgCCJvDatqhBDjgaUER5eWYvkFmywtCOcVtOYHOZIh dafwT5drZw4amTTWbd941b0av6PDtcD+WeDlKbztn2F41eQPMJBTbL5INpa9S5TTeE7l mD3cnnPat01GZjIwczXjBb8C7c5sbJCEtBlHwz1/isaYivCqiBuZGaE9r6wri1s2W9CR 9mw5ilU7LAeFVz5OdKr7hW4Jo64JWWa0UGKK7ESX9xr/ws1lKMWqH/7aql8Sr0pk+0TD s+jgRY8sgpI7fSLys7Pd81nQpqROsaK67GItSQrlp3zWRRGypyguO2dy9pOGnUxQj02A HtpQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id w20si677009ljd.2.2018.02.14.06.06.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Feb 2018 06:06:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w1EE6Pnx010363 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 14 Feb 2018 15:06:26 +0100 Subject: Re: [PATCH v2 0/9] Add support for automatic partial rebuilds on recipe changes To: Jan Kiszka , isar-users References: <96a5d7df-b7a3-08ec-3c28-85ecf332d32b@siemens.com> <01a34333-06a2-4df0-a51e-17aced9f9ee3@ilbers.de> <7bd0cc56-0dba-6a04-767a-de4edcc3be1e@siemens.com> <96dd5116-fc00-d5df-d48b-c91b6d332cae@siemens.com> <43c8ff54-68d3-e55c-6e0c-a2e60f53db30@siemens.com> From: Alexander Smirnov Message-ID: Date: Wed, 14 Feb 2018 17:06:19 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <43c8ff54-68d3-e55c-6e0c-a2e60f53db30@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: IgLDVjRPZewM On 02/14/2018 04:49 PM, Jan Kiszka wrote: > On 2018-02-14 14:38, Jan Kiszka wrote: >> On 2018-02-14 14:29, Jan Kiszka wrote: >>> On 2018-02-14 14:10, Jan Kiszka wrote: >>>> On 2018-02-14 13:57, Alexander Smirnov wrote: >>>>> On 02/14/2018 03:41 PM, Jan Kiszka wrote: >>>>>> On 2018-02-14 12:33, Alexander Smirnov wrote: >>>>>>> On 02/13/2018 11:05 PM, Jan Kiszka wrote: >>>>>>>> Yeah, finally Yocto/OE-like usability: This ensures for many cases that >>>>>>>> changes to recipes lead to rebuilds of dependent recipes, including the >>>>>>>> final image. Some extra measures are needed so that those rebuilds work >>>>>>>> with clean dirs. >>>>>>>> >>>>>>>> And if the change detection should not work, e.g. changes to file:// >>>>>>>> resources are not detected, then a clean or cleanall task is now >>>>>>>> available and ensures a proper manual rebuild. >>>>>>>> >>>>>>>> This massively increases the fun factor when developing Isar projects. >>>>>>> >>>>>>> >>>>>>> Wow, just noticed that with this series bitbake has started to run tasks >>>>>>> in parallel.. How it's working? :-) >>>>>>> >>>>>>> >>>>>>> Currently  4 running tasks (54 of 217)  24% >>>>>>> |######################################## >>>>>>>                       | >>>>>>> 0: mc:rpi-jessie:buildchroot-1.0-r0 do_build - 101s (pid 3271) >>>>>>> 1: mc:qemui386-stretch:buildchroot-1.0-r0 do_build - 101s (pid 3336) >>>>>>> 2: mc:qemui386-jessie:buildchroot-1.0-r0 do_build - 100s (pid 3457) >>>>>>> 3: mc:qemuarm-wheezy:buildchroot-1.0-r0 do_build - 100s (pid 3524) >>>>>> >>>>>> Didn't notice that this wasn't the case before (most of my workload had >>>>>> linear ordering), but I could imagine that the lacking signatures forced >>>>>> bitbake to serialize. Just guessing. >>>>>> >>>>> >>>>> There is still the same build problem as reported yesterday, log is >>>>> attached. Dropping patch #3 fixes it. >>>>> >>>>> Line I used to build: >>>>> >>>>> $ time bitbake multiconfig:qemuarm-wheezy:isar-image-base >>>>> multiconfig:qemuarm-jessie:isar-image-base >>>>> multiconfig:qemuarm-stretch:isar-image-base >>>>> multiconfig:qemui386-jessie:isar-image-base >>>>> multiconfig:qemui386-stretch:isar-image-base >>>>> multiconfig:qemuamd64-jessie:isar-image-base >>>>> multiconfig:qemuamd64-stretch:isar-image-base >>>>> multiconfig:rpi-jessie:isar-image-base >>>>> >>>>> Just for sure stated build in CI server: >>>>> http://isar-build.org:8080/job/isar_asmirnov_devel/8/console >>>>> >>>>> Also failed. Could I dropped this patch from the series? >>>> >>>> Not before you can explain why things fail and why that patch causes >>>> this. We need to understand the problem. Let me reproduce... >>> >>> Yeah, I get some bug as well, but a different & well-known one: >>> >>> dpkg: error: dpkg status database is locked by another process >>> >>> So, I'm afraid we will find more of these sporadic breakage, now that >>> parallelization works better.... >> >> BTW, I'm testing with >> https://github.com/siemens/isar/commits/jan/staging, i.e. the sum of all >> my misdoing. >> >> Good news is that I just reproduced your issue, now digging into what >> corrupts the isar-apt repo /wrt libhello-dev. Bad news: I had to restart >> 5 times due to the contention bug above. > > OK, understood this one: repo_clean. The arm task kills the results of > the i386 or vice versa. Need to work on the filtering rules here. Or we > need to split the isar-apt repos per multiconf target... This how ftp.debian.org is organized, the pool contains all the supported architectures for specific package: http://ftp.de.debian.org/debian/pool/main/liba/libapache-mod-auth-radius/ Alex