From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6727118917296193536 X-Received: by 2002:a1c:770d:: with SMTP id t13mr25481682wmi.27.1566297618246; Tue, 20 Aug 2019 03:40:18 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ab1b:: with SMTP id q27ls5388360wrc.7.gmail; Tue, 20 Aug 2019 03:40:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqxzoElPXDMM0MInso65r89NO7faqzJ0y5qsb+UVlR5mMrdJAS35kxWjegBaQeM0Ru6laVPw X-Received: by 2002:a05:6000:148:: with SMTP id r8mr22886281wrx.312.1566297617614; Tue, 20 Aug 2019 03:40:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1566297617; cv=none; d=google.com; s=arc-20160816; b=e0Vc50GhbXnkgyPxYYF8HlXMguxhaR+AFd+hrO9MhPAEVri6gKV0Tn8ZgiU1TWT+kc FkVkyCvFrl8kaTElW4RbXUoSHL7F5EIj0mPUsRVh5wY3XCCq8QIRfjElqz2Uc8+A3eOw KUqQZ77tR0rQBmZVR5zhM3s4wCBactDuz4LGB/10zpX2c7IEg1LLRQiIShgfGVPqmpLK 4zlW645vzHyfDLABfKQsT+SD03qm+FcMn9jzqf/SC6EeLAwckFrolHgqAwgJd2RqHj0M m3+Q9LKIQMKGqYSO1GDOWBpz7lphQv+ccLpGtB0NMVtsNrWQry5txgITbPLHtISJX6I3 v1iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=0DDT9THVtEcT5Q+dg2v205U8jdiyQQthSI1Y04aKO+s=; b=SM/Wr/nc2+dTtzu4wbIkTL5jbQkQ6hv84Ld5tm3Jucqd6eJ+44xd283ZG8hKCbRhno Vw3HC0jBr/FCz0ek2xVEbbea3Zjuv+Pd9O/4VMQidZz/C498DF2kfVcm1Pz7iJIocXeV 5Z+BcQqAK7L+Ira1H41WcDg7U4vJIwmJVuF9pzIfjP2IrNvDQEr8cJwa7aXlCCI1ux5R PYfkxRg7ISwQwu9faY/lxYcDN4rMrfluroXSPzAKwOrIwd5Uk6YZvoAlnQWPZh+5MsCT dNychTQIBwwM120EyLofyzY9InhszxzvSrxKAoCera+0RWUDNmTzxFyFjzbV795Lvmma bjaA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id g7si160581wmk.0.2019.08.20.03.40.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Aug 2019 03:40:17 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id x7KAeHP5030198 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 20 Aug 2019 12:40:17 +0200 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x7KAeHHk000737 for ; Tue, 20 Aug 2019 12:40:17 +0200 Subject: Re: Sporadic build failure of next To: isar-users@googlegroups.com References: <1caa3be8-d9c7-0f31-7cb7-4ee3ad43af27@siemens.com> <20190820103140.GI3412@yssyq.m.ilbers.de> From: Jan Kiszka Message-ID: <2d9c8c0c-f087-6940-89f2-72679405b886@siemens.com> Date: Tue, 20 Aug 2019 12:40:16 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190820103140.GI3412@yssyq.m.ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 4WUoI2NweR8y On 20.08.19 12:31, Baurzhan Ismagulov wrote: > Hello Jan, > > On Tue, Aug 20, 2019 at 07:37:04AM +0200, Jan Kiszka wrote: >> attached a build failure of 49895c62cdca. I ran that build 3 more times, but >> it didn't trigger an issue again. Looks like some false sharing, but I do >> not have an idea yet of what exactly. > > Thanks for heads-up. > > ERROR: mc:de0-nano-soc-stretch:libhello-0.1-98f2e41-r0 do_build: Function failed: do_build (log file is located at /builds/ebsy/debian/isar/build/tmp/work/debian-stretch-armhf/libhello-0.1-98f2e41-r0/temp/log.do_build.56623) > ... > ERROR: mc:qemuarm-stretch:libhello-0.1-98f2e41-r0 do_build: Function failed: do_build (log file is located at /builds/ebsy/debian/isar/build/tmp/work/debian-stretch-armhf/libhello-0.1-98f2e41-r0/temp/log.do_build.69020) > > Apparently, we instruct multiconfig to run the same task twice for the same > DISTRO and DISTRO_ARCH in the same directory, and they happen to run in > parallel. If this is the case, we should ideally build once; not sure whether > this is desirable for all packages, though (e.g., two boards building the > kernel from the same source and revision, but with different configs, etc.). > That usually happens when we feed in a variable into the build that makes the package different - without adjusting its name. > I couldn't find the same failure on ci.i-b.o, but I remember some issues around > this (related or not). > > Could we force building of the packages at the same time in CI? Ideas welcome. > In this way, we could perform regression testing for concurrency. I suspect we can when we first build multiple multiconf targets up to the level needed to build that package and then request to build it for all those multiconf targets. > > IIRC, the kernel or PREEMPT_RT has a static checker that can detect potential > deadlocks. That would be a nice addition to bitbake (although I doubt it is > implementable today, since it would require recipe introspection w.r.t. e.g. > build directory, etc.). Maybe evaluating the stamps after a build helps. There should be different pattern of a recipe was built multiple times for the same distro-arch. BTW, I'm about to adopt the OE layout for the stamps folder. Let's see what that brings (beside visual alignment). Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux