From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7211363025731715072 X-Received: by 2002:a62:a512:0:b0:5a8:a475:918f with SMTP id v18-20020a62a512000000b005a8a475918fmr5430965pfm.4.1679302311706; Mon, 20 Mar 2023 01:51:51 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:903:80e:b0:1a0:482d:27cc with SMTP id kr14-20020a170903080e00b001a0482d27ccls11026094plb.3.-pod-prod-gmail; Mon, 20 Mar 2023 01:51:50 -0700 (PDT) X-Google-Smtp-Source: AK7set95wdfA1NGVQSGQJkPKScTzUsbuoM7Nk3VioRtGngQHboqvl4hW8qgVJkjXyOGLIcrj6Xw4 X-Received: by 2002:a17:90b:314a:b0:23d:3a3f:950b with SMTP id ip10-20020a17090b314a00b0023d3a3f950bmr18417536pjb.22.1679302310747; Mon, 20 Mar 2023 01:51:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679302310; cv=none; d=google.com; s=arc-20160816; b=IgVFe+yowgBfHsoTkfP2OOsTz0IZ4qVZ86MG0sh0bz6iDufGLd1efERK5EIrl8wcai K/wGu3G/FKJlYLR5gZwL7SMzQ3AT7YKWIWft0sHe5/nqNjOpkoe+/mMEfRe7wPYcx6FA yVXJheWaQhxjutMr7922gRHL24G7Tn4kw3msVYs+SimczIrRIoSKA0D3jvr9v931RMCE MJarxkONtdb0k0mY33yDm+wmBUxFfnp7wG9x2clFH4W57KfayNKkQrQNfXMHxCr4dRgd 95ezKWjcp8FvaUpniz1/hptK8E4giS7tJtiMqcbbPb5xUl2qkbTOikD6z8YKdBgT3wvV XoLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=+RCPrWdAiikbWt2yS+7UAz/MmhDq8XSQ2O52hx3SPXY=; b=aO84GBwr7hbD4q7ZtQqG3ieXpz8OSC2mJgM3w+vd5HmZo4Wb8YjizN199D6SVjfQVx ijLHCylLbh9dTAnJwScCcs+fewVoImIwRAjg+1DTnlGlPHuVvCOqGJegCXMWlrjQLbPS 20/hphyxv0G5c+iScsuqELzuHiVz0qDmVFrmBY8e32c9wCzJVh/CkcIWmMF6xHDNjLkg fApY9Wl7q0ZXP/pg6pRNeCFggdqjkrQcChzAoRf30Rz9fNLmouxmAiW7m5U2Fruo1DUf INyAqCYwHNnuBadhNiLZh/1HLfEFjvBpjQ0wN7Np11U+/2PYt1lTLgZw59GgkS2LDF9H cNDg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id s15-20020a17090a6e4f00b0023f29444ab2si384896pjm.2.2023.03.20.01.51.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 20 Mar 2023 01:51:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from hp.localnet (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 32K8pjNb018961 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 Mar 2023 09:51:48 +0100 From: Uladzimir Bely To: Henning Schild Cc: isar-users@googlegroups.com Subject: Re: [PATCH] testsuite: Improve SSH ping Date: Mon, 20 Mar 2023 11:51:57 +0300 Message-ID: <2470465.XAFRqVoOGU@hp> In-Reply-To: <20230317095358.6a73547f@md1za8fc.ad001.siemens.net> References: <20230317041130.7987-1-ubely@ilbers.de> <20230317095358.6a73547f@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: Q5JNJxYj/JyK In mail from Friday, 17 March 2023 11:53:58 +03 user Henning Schild wrote: > Am Fri, 17 Mar 2023 05:11:30 +0100 > > schrieb Uladzimir Bely : > > When qemu machine boots, it may happen that consecutive SSH connection > > fails right after the previous good one. So, we get a situation when > > the command/script fails after we consider SSH is ready. > > > > This patch improves detection of SSH server ready status by making at > > least three good consecutive SSH pings. > > > > Example of debug output that shows the case: > > > > ``` > > > > | Waiting for SSH server ready... > > | SSH ping result: 255, left: 300s # <== machine is booting > > | SSH ping result: 255, left: 294s > > | SSH ping result: 255, left: 288s > > | SSH ping result: 255, left: 282s > > | SSH ping result: 255, left: 276s > > | SSH ping result: 255, left: 270s > > | SSH ping result: 255, left: 264s > > | SSH ping result: 255, left: 258s > > | SSH ping result: 0, left: 253s # <== SSH server is up... > > | SSH ping result: 0, left: 251s > > | SSH ping result: 255, left: 250s # <== but one ping failed again > > | SSH ping result: 0, left: 248s > > | SSH ping result: 0, left: 245s > > | SSH ping result: 0, left: 243s > > | SSH server is ready > > | `lsmod | grep example_module` returned 0 > > > > ``` > > > > Signed-off-by: Uladzimir Bely > > --- > > > > testsuite/cibuilder.py | 12 ++++++++++-- > > 1 file changed, 10 insertions(+), 2 deletions(-) > > > > diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py > > index 9e84c3a3..4e568b8e 100755 > > --- a/testsuite/cibuilder.py > > +++ b/testsuite/cibuilder.py > > > > @@ -257,17 +257,25 @@ class CIBuilder(Test): > > self.log.debug('Waiting for SSH server ready...') > > > > rc = None > > > > + goodcnt = 0 > > > > while time.time() < timeout: > > if proc.poll() is not None: > > self.log.error('Machine is not running') > > return rc > > > > rc = self.exec_cmd('/bin/true', cmd_prefix) > > > > + time_left = timeout - time.time() > > + self.log.debug('SSH ping result: %d, left: %.fs' % (rc, > > time_left)) time.sleep(1) > > > > if rc == 0: > > - self.log.debug('SSH server is ready') > > - break > > + goodcnt += 1 > > + # Let 3 good SSH pings to make sure SSH connection > > is stable > > + if goodcnt >= 3: > > + self.log.debug('SSH server is ready') > > + break > > + else: > > + goodcnt = 0 > > This looks like an endless loop should ssh never come up. Not sure what > would break that loop. In the worst case a test-timeout and everything > being stuck because we might not execute these things in parallel. > In case ssh never come up, we will exit after the timeout (default value 600sec), goodcnt will always be 0 in this case. Using something more readable like "for i in range(3)" you've suggested in the previous comment won't work because we need to reset the counter and start again in case one of pings goes wrong. E.g, if second ping fails, we need `goodcnt` changes like 0-0-0-1-0-1-2-3. With "range" function it will act like 0-0-0-1-0-3. But you are right, code like "while time.time() < timeout and goodcnt < 3" might be a bit more readable. I'll check it. > Henning > > > if rc != 0: > > self.log.error('SSH server is not ready')