From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7005854347145773056 X-Received: by 2002:adf:d084:: with SMTP id y4mr13618015wrh.249.1631627678100; Tue, 14 Sep 2021 06:54:38 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1:: with SMTP id g1ls870357wmc.3.canary-gmail; Tue, 14 Sep 2021 06:54:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxI6w8fRXhLj/4G2CqwypjOfdUGrbovHKhQ70eTMWqIwDK/qK1YRSKnX59/OEODmbZckQjQ X-Received: by 2002:a05:600c:4f52:: with SMTP id m18mr2345763wmq.34.1631627677037; Tue, 14 Sep 2021 06:54:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631627677; cv=none; d=google.com; s=arc-20160816; b=yq7MGhmLm7UWnMSkAZcrRtDTeAZnNizQAsd4v0HY+mZOtbZsN5xUWZHhofuvyFNEIj jnHuSKA9mNlm6N7jUgkwATwEnd+btxj11ZN9/AxNvntnIB5YyFXDyAeNp8Znn/Ly+CG7 byw0CKWEBhC9rZv46rCtsvf4glsqkKXs0wTdAerpCUC59CqVO1/tYQGKipaWjOjMrhBQ xz/WPfJdeINLSgG1ssrY3515qWJOVBy2ujKVG2LYKilcCJG0ZaIbmDNfzcRcZKDkktLN MUNqwCHp8UmiFoMtCjxC3tKaKtUwbjM4K1Tp/GLov6+ljBGGaEhmIpk/00GU1cx1wESo OIOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=ZcS5UdOA67plYzApD0HifxCeKJEgZy1ZRld3Bfp6kwk=; b=Nc9ljwyMufBU2+pkV6TwpD95rMEAW8yeplqRBJMJciWrpnJhJyoFnBwOIAAwlBQaz4 isD50TtKL9oBWTJT4FWeNsij+ttdfbqA7jf4Wj8carQRwPMB7N8K/Ef9GbvqhsCUVd5P SaYcYf7h8eCHnsYOOLEPyJ7H6w3U17J2suscq7ERIoKSo0vdGICsgnCRMPdzBGsx8sP/ EUt5ZekXKPi3d1aUP9y1Hg3guajHmeQ1SmsRFdTs7aq4kjcOV4A4UK6jnvf7FQ95HUl8 HXGNjxCf55wUL31YYcj2jaT4m2rQwkogXStgXpv3mcYZo25ZEWZX9NngUILtXZXomQKW y4Ug== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id o2si115555wme.0.2021.09.14.06.54.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 Sep 2021 06:54:36 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [192.168.67.164] (mm-240-58-214-37.mgts.dynamic.pppoe.byfly.by [37.214.58.240] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 18EDsXja014117 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Sep 2021 15:54:34 +0200 Subject: Re: isar next getting stuck, most likely on ("REPO_ISAR_DIR") + "/isar.lock" To: Henning Schild , isar-users Cc: Jan Kiszka References: <20210909105253.71df16d5@md1za8fc.ad001.siemens.net> From: Anton Mikanovich Message-ID: <18853bbd-9757-d1d2-90b9-eacf256640df@ilbers.de> Date: Tue, 14 Sep 2021 16:54:28 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <20210909105253.71df16d5@md1za8fc.ad001.siemens.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-TUID: y12e3BSElEc5 09.09.2021 11:52, Henning Schild wrote: > Hi all, > > we are seeing isar builds getting stuck when updating from > > b78bdb434ef619e13e5c51c74c023fe29c6b3fb6 > > to > dc36d449dc832db0164f088f3669889557ee015b or beyond. > > As of now we did not figure out which patch is causing the issue. > > We usually get stuck at some package wanting to do_deploy_deb and then > run into a CI timeout. > > 2021-09-08 16:56:07 - INFO - NOTE: recipe > linux-perf-5.10-5.10.26-r0: task do_deploy_deb: Started > 2021-09-08 17:43:09 - INFO - Bitbake still alive (5000s) > ERROR: Job failed: execution took longer than 3h0m0s seconds > > One one occasion we logged into that CI runner and simply deleted > ("REPO_ISAR_DIR") + "/isar.lock", which got the build un-stuck. > > Investigation is ongoing, just to let people know. Maybe others have > seen that as well. > > Henning > It looks like this information is not actually correct. After looking into downstream CI I've found at least 15 stuck builds. Most of them were actually using b78bdb434ef619e13e5c51c74c023fe29c6b3fb6 But there also 3 other hashes used by those stuck builds: 5e563ac4b7072151e6013a8f516c2e346e8677db dc36d449dc832db0164f088f3669889557ee015b e274130b870b7e31532fe191b9932cde1d819b4b The first time it was observed was 2021-08-18 14:33:23 (53199018) with Isar on b78bdb4 The last one - 2021-09-10 19:16:49 (54858872) also with Isar on b78bdb4 So it is definitely not something started after switching from b78bdb4 to dc36d44, but something happened before. -- Anton Mikanovich Promwad Ltd. External service provider of ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn, Germany +49 (89) 122 67 24-0 Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov