From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7211363025731715072 X-Received: by 2002:a05:6871:23c5:b0:17a:b713:63e9 with SMTP id xy5-20020a05687123c500b0017ab71363e9mr2221344oab.4.1679303861495; Mon, 20 Mar 2023 02:17:41 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6870:c227:b0:177:90a9:cbc3 with SMTP id z39-20020a056870c22700b0017790a9cbc3ls4072952oae.2.-pod-prod-gmail; Mon, 20 Mar 2023 02:17:40 -0700 (PDT) X-Google-Smtp-Source: AK7set83hRerO6Yz6fFYQ6EdVY73uMq5ioiG9w+jVsheyg7fXXntB1uuNVncrHXA1NMTsZ6xLo6r X-Received: by 2002:a05:6870:20e:b0:177:9a66:2a82 with SMTP id j14-20020a056870020e00b001779a662a82mr4549177oad.51.1679303860875; Mon, 20 Mar 2023 02:17:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679303860; cv=none; d=google.com; s=arc-20160816; b=XIQ2vf2eQ/F7m1jJuLE9ArqnroZowd3qFk1SeSU0fm5pTGBCX+XAI7nnKqwZo1L8cH ji/PR2q9s5tEeGN76ek8kfKKWxYCKh5bEOFdmJN+lLK6Vs0u8VdFDh3InYDxBLMje4dv ++5joQDXE+unDBJmlWjoiIFsEZnZMEYJTyWKn6Mnjz+VtfAprXB1J12X5sijmTpY3ZYs QnpHJlJPxScJVN3GQURXrclFbhlZc971Sm1WuK+1GTi8WPOHoJkangRn7exFbYW9N0od 5QuvsGEsTFTfuJ0T6aCVHlRHL2xmZeD+gTrvfcxAp1CPyAi+OMdsA6BFYNJpVahaX0Tf Wx0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=4o4e3ptKcmS92KDqQQn5yiAGd+Ojl3Uh/qmmcnYb7NI=; b=fb65mG6Fg9ZTh1V5O/qjf9P2N6o2ozjjB8lil0QJvrt1HWYwR7iQ/yuhFb0xmGhN/B MlUr7OXKujt++9g//C27Dq/wtl5wXT5PJjZ4iMD8MWqX1HU2brcOFOuPtsz97FCfcmCn SYUyBCC5ZAFsAkEVJAo1nVECqpiP1IW4msqQBHz79riCj39HWkl3l9qBGed/vXRoMvfu PjEuHxKptOyyde/xzX8LAn6bJEpQlPF+20yK/2WGtBmVRJ6MvplLeUirNU0ZnWYz5CGr iqlBuZi9tQWbmob8B6oMQHj5ZBkTUqfG1ad2dVfej9Vvg0Id6LxxGLSyZbVg2EYbZPiS c8dA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id oi15-20020a0568702e0f00b0017799b87aebsi1080371oab.1.2023.03.20.02.17.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 20 Mar 2023 02:17:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from hp.localnet (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 32K9HbU9019141 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 Mar 2023 10:17:38 +0100 From: Uladzimir Bely To: Henning Schild Cc: isar-users@googlegroups.com Subject: Re: [PATCH] testsuite: Improve SSH ping Date: Mon, 20 Mar 2023 12:17:49 +0300 Message-ID: <1912924.7Z3S40VBb9@hp> In-Reply-To: <20230320101243.1cea9ff5@md1za8fc.ad001.siemens.net> References: <20230317041130.7987-1-ubely@ilbers.de> <2470465.XAFRqVoOGU@hp> <20230320101243.1cea9ff5@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 10QEES+/c44U In mail from Monday, 20 March 2023 12:12:43 +03 user Henning Schild wrote: > Am Mon, 20 Mar 2023 11:51:57 +0300 > > schrieb Uladzimir Bely : > > In mail from Friday, 17 March 2023 11:53:58 +03 user Henning Schild > > > > wrote: > > > Am Fri, 17 Mar 2023 05:11:30 +0100 > > > > > > schrieb Uladzimir Bely : > > > > When qemu machine boots, it may happen that consecutive SSH > > > > connection fails right after the previous good one. So, we get a > > > > situation when the command/script fails after we consider SSH is > > > > ready. > > > > > > > > This patch improves detection of SSH server ready status by > > > > making at least three good consecutive SSH pings. > > > > > > > > Example of debug output that shows the case: > > > > > > > > ``` > > > > > > > > | Waiting for SSH server ready... > > > > | SSH ping result: 255, left: 300s # <== machine is booting > > > > | SSH ping result: 255, left: 294s > > > > | SSH ping result: 255, left: 288s > > > > | SSH ping result: 255, left: 282s > > > > | SSH ping result: 255, left: 276s > > > > | SSH ping result: 255, left: 270s > > > > | SSH ping result: 255, left: 264s > > > > | SSH ping result: 255, left: 258s > > > > | SSH ping result: 0, left: 253s # <== SSH server is up... > > > > | SSH ping result: 0, left: 251s > > > > | SSH ping result: 255, left: 250s # <== but one ping failed again > > > > | SSH ping result: 0, left: 248s > > > > | SSH ping result: 0, left: 245s > > > > | SSH ping result: 0, left: 243s > > > > | SSH server is ready > > > > | `lsmod | grep example_module` returned 0 > > > > > > > > ``` > > > > > > > > Signed-off-by: Uladzimir Bely > > > > --- > > > > > > > > testsuite/cibuilder.py | 12 ++++++++++-- > > > > 1 file changed, 10 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py > > > > index 9e84c3a3..4e568b8e 100755 > > > > --- a/testsuite/cibuilder.py > > > > +++ b/testsuite/cibuilder.py > > > > > > > > @@ -257,17 +257,25 @@ class CIBuilder(Test): > > > > self.log.debug('Waiting for SSH server ready...') > > > > > > > > rc = None > > > > > > > > + goodcnt = 0 > > > > > > > > while time.time() < timeout: > > > > if proc.poll() is not None: > > > > self.log.error('Machine is not running') > > > > return rc > > > > > > > > rc = self.exec_cmd('/bin/true', cmd_prefix) > > > > > > > > + time_left = timeout - time.time() > > > > + self.log.debug('SSH ping result: %d, left: %.fs' % > > > > (rc, time_left)) time.sleep(1) > > > > > > > > if rc == 0: > > > > - self.log.debug('SSH server is ready') > > > > - break > > > > + goodcnt += 1 > > > > + # Let 3 good SSH pings to make sure SSH > > > > connection is stable > > > > + if goodcnt >= 3: > > > > + self.log.debug('SSH server is ready') > > > > + break > > > > + else: > > > > + goodcnt = 0 > > > > > > This looks like an endless loop should ssh never come up. Not sure > > > what would break that loop. In the worst case a test-timeout and > > > everything being stuck because we might not execute these things in > > > parallel. > > > > In case ssh never come up, we will exit after the timeout (default > > value 600sec), goodcnt will always be 0 in this case. > > > > Using something more readable like "for i in range(3)" you've > > suggested in the previous comment won't work because we need to reset > > the counter and start again in case one of pings goes wrong. E.g, if > > second ping fails, we need `goodcnt` changes like 0-0-0-1-0-1-2-3. > > With "range" function it will act like 0-0-0-1-0-3. > > It is a nested loop, obviously the outer one needs to be changed a bit > as well. But the inner can maybe be done with range. > But there are no currently any nested loops, there is only "while". Or you mean it should be? > Henning > > > But you are right, code like "while time.time() < timeout and goodcnt > > < 3" might be a bit more readable. I'll check it. > > > > > Henning > > > > > > > if rc != 0: > > > > self.log.error('SSH server is not ready')