From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6739560601010307072 X-Received: by 2002:a17:906:8258:: with SMTP id f24mr18029561ejx.234.1569823224204; Sun, 29 Sep 2019 23:00:24 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:d7a3:: with SMTP id pk3ls2581179ejb.3.gmail; Sun, 29 Sep 2019 23:00:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqwIctSa+Wd6erICrANbn0iyqhGVuAgESW13o2fFarUwsUbe2UzTDuBpKc2P+ck8cch5zqnA X-Received: by 2002:a17:906:6bca:: with SMTP id t10mr17990309ejs.64.1569823223674; Sun, 29 Sep 2019 23:00:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569823223; cv=none; d=google.com; s=arc-20160816; b=pCiezB0+NGv1oQE0YHjwCh9gsk8j4MgtY8JAQ4aF2WdweDST2XQDjqIoc2q9G8IjMf p8jIFcRZGHTvgrtMheG11hTN+XrDOZh+ztSF1R9EuzHF1Rsc6SmYrv6F9RY87LxiADCb bqLaUBbzoM7sXGYfEiUC5gLjc7zgQvBTUVoGiUf0fW3G7j0YaSr7nWSNyjQMAndTcaj+ 32MLlsfFjcdnA7q1YMycBK4IjXn2bMMabkd9TSUNWM8W8pVoAUXRLi7BAF9UlJI0OHjM XMJl1exGs/0Ni6n8u8hIiFuCCsuRYDS8U9OLUILL8HI3fKpHojJ/TwpAtO0Go54tuu5H C2NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=mcbvH3kIXFQnIKvqPEsgZnO0Cnku8FXNl5NvXZpmDxg=; b=cowt1GBMakk7uqBbEcDLB+QpU5W2AbRlQWIGdK0TFgR4RS+okei2jiM4OKJyWLQsB6 mmMJRIhMsNMOhfHns3HjTR98dyiUlU0J7xLFOajlV7iJszBgDdBE/deLnXufY8T6o+xW 52hyHys79Bj3ilG+imi/LIsULpPQ+NCmav37CY/82Z35iS7WpD5hCGzJEbFzI3fsqp9k TIM0A3JmOT/HGoOLsEE2FPj6BYL4+Zwb/xDcoJZZxg7/jCfqJTlWZlya0LtiuEMZwrqd h0scHuxvfaOrCn4o3mwcZKgDb2g+rga/Mj27qks6lNCD2uCKGLyla9dTLiR2KxJAduTJ QBpQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id q8si562672edn.5.2019.09.29.23.00.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Sep 2019 23:00:23 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id x8U60NKr029902 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 30 Sep 2019 08:00:23 +0200 Received: from [167.87.40.9] ([167.87.40.9]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x8U60Mbr021239 for ; Mon, 30 Sep 2019 08:00:22 +0200 Subject: Re: [PATCH v6 12/27] Detect false sharing of recipes To: isar-users References: <5a2e329b881ec0b392d0f1abd116f1deeee0f66f.1569176231.git.jan.kiszka@siemens.com> <20190929145700.ads3zbq7hz77olcw@yssyq.m.ilbers.de> From: Jan Kiszka Message-ID: <248aa6db-6327-9604-40b6-e7135a5f9961@siemens.com> Date: Mon, 30 Sep 2019 08:00:21 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190929145700.ads3zbq7hz77olcw@yssyq.m.ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: Fg29jfLtGSqG On 29.09.19 16:57, Baurzhan Ismagulov wrote: > On Tue, Sep 24, 2019 at 08:02:26PM +0200, Jan Kiszka wrote: >> When a task is run more than once per build, this indicates a bug, >> usually (though not only) related to incorrect multiconfig use. > > Do I understand correctly: > > 1. Multiple isar-bootstrap executions (apart from the fact that they are > unnecessary) could fail because some steps after debootstrap (which was > locked) were not using the lock and could overwrite each other's results? > The background is to understand the specific failure mode, because you wrote > that this locking didn't actually protect from races. > > 2. Multiple isar-bootstrap executions do not occur now because the same tasks > that should not be executed twice have the right PF? > isar-bootstrap had two issue: 1. Not rebuilding when it should (on changes to its configuration) 2. False sharing of it deployment directory None of them were or could have been detected by this instrumentation. What would have been detected AFTER removing that improper rebuild prevention (https://groups.google.com/d/msgid/isar-users/ccad9eb9-fce5-89d1-3188-7043f003ea7e%40siemens.com) is https://groups.google.com/d/msgid/isar-users/ccad9eb9-fce5-89d1-3188-7043f003ea7e%40siemens.com - which is still not in your queue. Lost in your staging tree? I removed it from my last repost because it was staged already. > >> This adds a very simple but effective check for such re-run scenarios >> by putting a "once." stamp file into each workdir during a >> build. If such a file already exists, an error is thrown. The workdirs >> are purged after the parsing phase on the beginning of each build so >> that no false positives are raised. The task stamp is also deleted after >> failures so that no (serialized) retries can cause warnings. > > 3. Where does this delete the once.* file w.r.t. e.g. do_dpkg_build for > different multiconfigs? > At the beginning of the new build (parse_completed) or when a recipe failed to build (task_failed). Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux