From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6521574336665485312 X-Received: by 10.28.87.5 with SMTP id l5mr298132wmb.23.1518550745737; Tue, 13 Feb 2018 11:39:05 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.199.10 with SMTP id x10ls1829797wmf.9.gmail; Tue, 13 Feb 2018 11:39:05 -0800 (PST) X-Google-Smtp-Source: AH8x225f/9/lGaacotDe1K0RExVtZT1iOuHHj9Lp5Gzi50IM5OwZ0U+dnEpVVnBypXHWZ0vVBid1 X-Received: by 10.28.89.193 with SMTP id n184mr292538wmb.22.1518550745334; Tue, 13 Feb 2018 11:39:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518550745; cv=none; d=google.com; s=arc-20160816; b=jZ/WoO3guP5FGo3btU89OLh89PQFpOeQIAqx+0wgx/bqLI/M9F8wqZBeaR8Hz1dwHX lMyKqOPZq70g2eR6A6SqWUqkVK8pqr2WSSAqIpLM7rPsU6ePaxuoNHvcVk3oy7lmlqhG 4fI13qsWdDKbNr5IjKFeXio/Ls2NA4sWTDeBrNUnjTK0dTITd9t5x8tCd4A5IlY4bbm6 lGmsGXlTWbx4eCvX6ePlHlff3nnPDuImK4SptOjk7+QSzXfPnrnzNQrX4zavnai2v1oQ UhpL7HlF8LDttMpYrZkxp8lT8tY7mcAkBfJYisuldAmFc/ijMpLPq0BnsTzpJGlhr1pT tkMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=ZulYPREB9kvtNgmHdfgEdxzh2pTzk2YMDmwkIc4RfFw=; b=URjUIK1Y7ZZZeYVg4KLZNGPmk/PGoezi0jFzW2a0XOhe6BskuFczdYzaiud5roVc2h h0qOXDFBU6uehCPm0KXYDr0IebTRJ6yUL8CVG0p8v+22eIBvz23rUk1kxBUm+gR/IAYk HWx8aAov7q0xLh794lXpYvtNaW6x7CmJZsjKRVfyKkexwLP34Rvbjyl9Eesti3j87K33 DErxvpFnaLqJB01/FRa3fj3sUOBo1gpdh156cR2elCXtNzZSokay2aIojs0DBm/TwDe2 Rr7LSQTgp76hwXaZqNawj5vSjMnZ2uyc3dzYK+cYdYv71gHT7I8ussfZyU+obgPmdfih 8fhA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id t15si30990wmh.1.2018.02.13.11.39.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Feb 2018 11:39:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w1DJd29P007240 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 13 Feb 2018 20:39:04 +0100 Subject: Re: [PATCH 4/6] Enable proper rebuilds on dependency changes To: Jan Kiszka , isar-users References: <7d0504da7640612abded5ffe1e1b3c6d10168b49.1518422347.git.jan.kiszka@siemens.com> <73bd323b-0dc2-a84a-ae5a-59177cfe6921@ilbers.de> <4b84a0e3-df9e-8343-ff09-7b74474a0e95@siemens.com> <20278b99-b41b-ee89-b5cd-eafc2019875d@siemens.com> <7c44164d-bae8-82d3-e9c3-bb60918bfd60@siemens.com> From: Alexander Smirnov Message-ID: Date: Tue, 13 Feb 2018 22:38:57 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <7c44164d-bae8-82d3-e9c3-bb60918bfd60@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: WdrqsV0Go/h9 On 02/13/2018 10:22 PM, Jan Kiszka wrote: > On 2018-02-13 20:02, Alexander Smirnov wrote: >> >> On 02/13/2018 09:44 PM, Jan Kiszka wrote: >>> On 2018-02-13 19:08, Alexander Smirnov wrote: >>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote: >>>>> On 2018-02-13 17:21, Jan Kiszka wrote: >>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote: >>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote: >>>>>>>> From: Jan Kiszka >>>>>>>> >>>>>>>> Install a basichash as signature handler and set the stamp policy to >>>>>>>> full - and suddenly we get proper rebuilds of the image and all >>>>>>>> affected >>>>>>>> packages when some recipe down the dependency chain changed! >>>>>>>> >>>>>>>> We are using the legacy bitbake mechanism here as we do not have >>>>>>>> setscene machinery like OE. Still good enough for us. >>>>>>>> >>>>>>> >>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it >>>>>>> didn't work properly, not all the dependents were rebuilt. If it >>>>>>> works, >>>>>>> it's a very good feature! >>>>>> >>>>>> Yes, I'm using it since this weekend for both Isar as well as >>>>>> jailhouse-images development, and it helped a lot already. The only >>>>>> limitation I found, but I do not remember right now if that isn't >>>>>> inherent to bitbake, is that it does not detect changes in files that >>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c >>>>>> clean" for the affected target - which now works as well. >>>>>> >>>>> >>>>> Found another issue, which worked surprisingly well so far despite >>>>> being >>>>> fairly broken: do_setup_mounts is not re-run on rebuilds. >>>> >>>> Investigating exactly this issue. It occures for clean builds also. My >>>> guess is patch #3, currently I'm building with only 2 first applied. >>>> With 3rd one applied build failed. >>>> >>> >>> That's not the right approach, the concept of the setup_mounts task is >>> broken. >> >> Why you think it's broken? Just tested, it works good: > > It's broken in its design: you stamp a task and then remove that stamp > again on restarts. That only works because we didn't generate the full > dependency chain. Once you do, all dependent tasks we be kicked for > re-run. Removing patch 3 is voodoo debugging. I bet you can't explain > why that works around the issue, and I'm sure it will break eventually > again. Moreover, it's needlessly complex. CI was green, I apply this series, CI became red, I've reset the head patch by patch to get CI green again. After reset patch #2 CI became green. Which voodoo you found here? This series brakes the build without re-run. > > The much simpler solution is to try-mount when needed, at user side. > Patch will follow, likely by resending the whole series Mount when needed like you proposed could lead to lots of mounts on the same folder, because do_build works in parallel. The correct way in this case would be: - keep do_mounts - use [nostamp] = 1 - use [lockfiles] = "${BUILDCHROOT_DIR}/lock" - mount folders like you proposed: + grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts && + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt || true Alex > > Jan > -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov