From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6521574336665485312 X-Received: by 10.80.230.18 with SMTP id y18mr917255edm.2.1518552268314; Tue, 13 Feb 2018 12:04:28 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.215.80 with SMTP id i16ls1364530edj.5.gmail; Tue, 13 Feb 2018 12:04:27 -0800 (PST) X-Google-Smtp-Source: AH8x226gMOfpDJVVjZMa+tx1idA/X2jgC3hBi3BMAPhKc2dHYQK1VYT8byR7insGtbJw9JUPhufr X-Received: by 10.80.146.139 with SMTP id k11mr912454eda.4.1518552267644; Tue, 13 Feb 2018 12:04:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518552267; cv=none; d=google.com; s=arc-20160816; b=Mq5qatsDv8Iq0P3EMTdMAwFfv1qs4naHZPLGMAtHdVt7IWZasUxi0ibTmy8b7YyGi/ luySGGlWP7hk7gmfBLvwfXATKPPLaLHGU4ROn9KUF8H/4BCgA91DeZtiF1ETtKf41vCt +XsOj7jh20taJWui0myesoQWhvFxOxLiXeYZCkN6e0l8ZZYMNFAPNROMGKf9TWzEEvaB dJH7ahTGH7vm+UNk2pYoUKld37GUjMCjvLfREKmH4ezsSXXBGWmCkLtqoFpBDwTTMutx /CVoGYgLNYPJV6zrCLf1wqjmprP/gW1/ovpdgyNiK6PcZqzBmT3LOFxrXoHpTZUbjo1u fGkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=/r97vvLb6eF5yvGBy6WKcJoD3FKVxHaw/Mb49MRIZwY=; b=VLYqaJGaejDnylr4Q8GNuzL53+JBjC5RskRIrI+whf+h1g0lC9183KNmPeawIozZz6 eJuxcSW0bUgE3Oy4+KjX3yIRcOyswBaeAPa5BKj6arSzmFwgX31voPwIG5+6XMTydxLa b18owataAWwvtj5DeOTDvKirGELp8d8KTkbjHTet5471lAJ4uj7w/AVDrgrpBp3GtJpu OJmmbpB5/V72SA7oXFTkLzH+hbo9ku2DstYWnKsuSXJFpN+PwDOv7TC1RS+p3ii6rRO5 8VhqV4X9FyZFZQlSTcf5eLP0MVznB3hqLbyKYOLJBcy6aCzhH5ZMB+7H9Z2Kz3LRXfI9 DcPg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id n22si471856edb.5.2018.02.13.12.04.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Feb 2018 12:04:27 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w1DK4RNM017080 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2018 21:04:27 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w1DK4QMn030913; Tue, 13 Feb 2018 21:04:26 +0100 Subject: Re: [PATCH 4/6] Enable proper rebuilds on dependency changes To: Alexander Smirnov , isar-users References: <7d0504da7640612abded5ffe1e1b3c6d10168b49.1518422347.git.jan.kiszka@siemens.com> <73bd323b-0dc2-a84a-ae5a-59177cfe6921@ilbers.de> <4b84a0e3-df9e-8343-ff09-7b74474a0e95@siemens.com> <20278b99-b41b-ee89-b5cd-eafc2019875d@siemens.com> <7c44164d-bae8-82d3-e9c3-bb60918bfd60@siemens.com> From: Jan Kiszka Message-ID: <939ee5ae-0dee-8a19-579c-41c9dc40b928@siemens.com> Date: Tue, 13 Feb 2018 21:04:26 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: u39XuNQ3L4we On 2018-02-13 20:38, Alexander Smirnov wrote: > > > On 02/13/2018 10:22 PM, Jan Kiszka wrote: >> On 2018-02-13 20:02, Alexander Smirnov wrote: >>> >>> On 02/13/2018 09:44 PM, Jan Kiszka wrote: >>>> On 2018-02-13 19:08, Alexander Smirnov wrote: >>>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote: >>>>>> On 2018-02-13 17:21, Jan Kiszka wrote: >>>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote: >>>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote: >>>>>>>>> From: Jan Kiszka >>>>>>>>> >>>>>>>>> Install a basichash as signature handler and set the stamp >>>>>>>>> policy to >>>>>>>>> full - and suddenly we get proper rebuilds of the image and all >>>>>>>>> affected >>>>>>>>> packages when some recipe down the dependency chain changed! >>>>>>>>> >>>>>>>>> We are using the legacy bitbake mechanism here as we do not have >>>>>>>>> setscene machinery like OE. Still good enough for us. >>>>>>>>> >>>>>>>> >>>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it >>>>>>>> didn't work properly, not all the dependents were rebuilt. If it >>>>>>>> works, >>>>>>>> it's a very good feature! >>>>>>> >>>>>>> Yes, I'm using it since this weekend for both Isar as well as >>>>>>> jailhouse-images development, and it helped a lot already. The only >>>>>>> limitation I found, but I do not remember right now if that isn't >>>>>>> inherent to bitbake, is that it does not detect changes in files >>>>>>> that >>>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c >>>>>>> clean" for the affected target - which now works as well. >>>>>>> >>>>>> >>>>>> Found another issue, which worked surprisingly well so far despite >>>>>> being >>>>>> fairly broken: do_setup_mounts is not re-run on rebuilds. >>>>> >>>>> Investigating exactly this issue. It occures for clean builds also. My >>>>> guess is patch #3, currently I'm building with only 2 first applied. >>>>> With 3rd one applied build failed. >>>>> >>>> >>>> That's not the right approach, the concept of the setup_mounts task is >>>> broken. >>> >>> Why you think it's broken? Just tested, it works good: >> >> It's broken in its design: you stamp a task and then remove that stamp >> again on restarts. That only works because we didn't generate the full >> dependency chain. Once you do, all dependent tasks we be kicked for >> re-run. Removing patch 3 is voodoo debugging. I bet you can't explain >> why that works around the issue, and I'm sure it will break eventually >> again. Moreover, it's needlessly complex. > > CI was green, I apply this series, CI became red, I've reset the head > patch by patch to get CI green again. After reset patch #2 CI became > green. Which voodoo you found here? This series brakes the build without > re-run. > >> >> The much simpler solution is to try-mount when needed, at user side. >> Patch will follow, likely by resending the whole series > > Mount when needed like you proposed could lead to lots of mounts on the > same folder, because do_build works in parallel. All solved, in a single line. Hold your breath, it's getting better. Jan