From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7211363025731715072 X-Received: by 2002:ac2:41cd:0:b0:4d5:ca43:704a with SMTP id d13-20020ac241cd000000b004d5ca43704amr6654181lfi.13.1679303574581; Mon, 20 Mar 2023 02:12:54 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:b29:b0:29c:88a4:18d0 with SMTP id b41-20020a05651c0b2900b0029c88a418d0ls426886ljr.0.-pod-prod-gmail; Mon, 20 Mar 2023 02:12:53 -0700 (PDT) X-Google-Smtp-Source: AK7set8nhJQI7s5vHg8tqyWgXATeSzoWfWp1oFHVrkwwpqHryHp6qWf2U3BnxUlZePZeDUbWuOGS X-Received: by 2002:a05:651c:1a0a:b0:298:aada:99a4 with SMTP id by10-20020a05651c1a0a00b00298aada99a4mr6907727ljb.15.1679303573131; Mon, 20 Mar 2023 02:12:53 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1679303573; cv=pass; d=google.com; s=arc-20160816; b=SJpHu3bZWVt+8+0Hb8V2Wzqg1UXBNPstbGf10+1ir9GtAUVlfVJoX5CH7ZU+MF+/Y4 mYSwDVmch6qMj7uXEtZTI5HYwMmwhJS8qHYkOjz5kNEINfesxXarg2eLfMAhyKEJ5VMK tTjSm6JFtxuDm3pzpr2giANPLjAxV3S/bJYoy++Nq1UwArBz+poU0YG/vY+fzRqGmF7c GG3MrR2IueelFBbF8b6P+qhIf8WrLXekq2ZkCaAzCKRiqQnRJaWRWQoKRMaGkpTcNOEV MDSr24JrbmBbf/ILQ5jjf3vxIODW+tzfHZSna+sYeCvZm9u7t4AOzvbM/tPM2WcKG2eO pGAw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=w3dnsAuPi1ruOeGXQpw1ff1XcYJrFFXFeEEtkB3Vqe8=; b=xnDUFQoEvK6rFufotLuY2oYnY8egsPhW78cB6gFjTW0sGYlxe4s1jz6SD9MBr64ueX oJhWytCSDo4La7HaX8Yf+mdvaFrQbnKhNHVTRtrg9Q4ORTqDT4y9B58g1Exe/mDpIVFO ftJeCQyo+ssgRYM8fLn0U59E8DLcqqnn8Bra7Sg9QmjEzjFOwHSawadSTsyI0LKCPrd4 fOeOu1iq+h6fRsOXuFxRHBPJaq12JLBQ8+P3eOCiNTFeRYsFQ09Zn8gsZpaN4M+lz/Iy zWQZi4tU5Jhw2LX4QQZJc8Rpm/Eb/fTz/3Qd5RpURq3fouR+FQSRKh/vk5g6Tge48Pc0 HodQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=Ow0utr46; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0c::629 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04on0629.outbound.protection.outlook.com. [2a01:111:f400:fe0c::629]) by gmr-mx.google.com with ESMTPS id be9-20020a056512250900b004e83bb20554si448177lfb.3.2023.03.20.02.12.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Mar 2023 02:12:53 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0c::629 as permitted sender) client-ip=2a01:111:f400:fe0c::629; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=Ow0utr46; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe0c::629 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j6JUUD0wqg6+PzWs/MYU30YHdBcbw+Zm5p9eX8lSCuDAaf0o8I1JywxaZwDx6MvEz2wPfq+yNMpu9E2CWZeUGskaYLExgUfuMnBiutjF1Y/vR4gI6AEjwNsRRnOyOzY3PHBJEQFugftHGU4aR6yxlW5RtUhv2kvgjebxtLXf8KZnzMFUY44iU8U5oBUMtQ1B4JU+T71m1O5/8qZsWQhcR9wuR/susnt+KTsRrEuWliYm8R84wRvwP/B2NlKo2Mmnp6fWTJ+/a8Vq9bgWRp/t8B9JU4vMOiM5Da+q0QPbVeHHvdP9uBQAzvjz0uLhR2XE5BQ6Rh/8BEvD0jvYT09iqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w3dnsAuPi1ruOeGXQpw1ff1XcYJrFFXFeEEtkB3Vqe8=; b=BhMRV9vgNxxyxcIEfw48SvxwOgI56JvXRTj5a7ajHZbCvFb9ku7PjvPpEIa0Cu4FbvFbg/i4Cp7DEZwNQsvXrrk4WmwYMQwRFF/sgTuwON16xvJ+m5BW1k8gwhiEpJBhyxmoyKMgw9jaoQD+zFkzMcUPY42e7skPX2m3is/Isz+IYEgpnept2wtimCr4nw6XWTl9t+w9mNjUlb6hcsTWjwrE9E7qHjnR5RyQLyDI4igQdsnEqK3eyebqFAwVvp2CLk98DMyX3IIhvyTejQCgDpLqJAFP6buFHZPLtv7QEhGTMI4oEhWVG7ZsPMsHBrZ11PZCwa9JuN7Az3bpHpImxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w3dnsAuPi1ruOeGXQpw1ff1XcYJrFFXFeEEtkB3Vqe8=; b=Ow0utr46NyXtveSdltWCmczjevlpNidBfoxGXy3VhlS/1s2PzeKYK7Aq62IenWL60d6ZuTZiXX9d06aIImplgCWKn4fK3XJkt62hkQ2Am435Z1ZBj2SdgkWqqjU9O28jKLV3FsJq7pea0spTLMMyu3RerWBYxNeuaFfeAbY0r2As9OMYwaI7T42BdIHI31vK52bnKVeCcNWqPG9BpBQyUeKbt80RDLBfY9KEszyFDM51js9TiOHFPRz002t7+fsLJAhupvlDs6sl0MALk442fi0x6XK93i3Ja5l8s/C4AwXIKEaaKrccfN99gLPtmeKX5EqixZxihHWW11USjHUDeQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by DU0PR10MB5312.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:34f::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6222.8; Mon, 20 Mar 2023 09:12:51 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::f777:d35e:83f2:6148]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::f777:d35e:83f2:6148%4]) with mapi id 15.20.6222.010; Mon, 20 Mar 2023 09:12:47 +0000 Date: Mon, 20 Mar 2023 10:12:43 +0100 From: Henning Schild To: Uladzimir Bely Cc: isar-users@googlegroups.com Subject: Re: [PATCH] testsuite: Improve SSH ping Message-ID: <20230320101243.1cea9ff5@md1za8fc.ad001.siemens.net> In-Reply-To: <2470465.XAFRqVoOGU@hp> References: <20230317041130.7987-1-ubely@ilbers.de> <20230317095358.6a73547f@md1za8fc.ad001.siemens.net> <2470465.XAFRqVoOGU@hp> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR0P281CA0129.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:97::15) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|DU0PR10MB5312:EE_ X-MS-Office365-Filtering-Correlation-Id: 0847b6fe-acb7-4270-d0bf-08db292345b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BKUdrGdPscbsUuiwkTCkkmWOXrWpzOuE/45hmB+Y53TRM0fELZLyhqTE0/maLY/0t65O3Qwu32YHKD5LAaIOGHX/pAev5drloNHbwRUxXgYPijFysBaWPI3akk1TJupnNp4P3vbTOLIDOrEJ9GJJnNabPda4LEGbkixoozQldvMIC7DDFA+8x41Cia9Wc7+DCMriQF6hJFMnFeWJKVP4ow+L/eZVCovmMgrvORm319ecbYxL9syo9ulIsWoz/GZDfx4b1rBJxoXWGecLTwPDbriDPbMP+YkdPdDp7Y4/XwpY26xKx4RFsQLLy+Q7m0UM7Zs1LdxEBSch2biWxPATtEwXkoQnnFumier8TQhusN5k2k3VpeRfPYbVo5WN9z55QVJsPGRWpiPad/N8tfoRv06oiczeSClKWY0vdxJ+WZbTDjae9/fJY5ErSObSelPMN4BV2NbuOkt2WH5AmiOuM/LhghJ9WkyR+RfdpPTih0E+qlrpNfPdkL36Vn9dLEA7JJkLZhyi4v0o/cZFjsGV9zYkb9OBpeoAcpBzlUVMGC/nKdLQOdO6wMl5R5qWt8T5EyZepW8JOXKXUH0yIi1ZnsHCOh2068AJuv3DbrMwTaflrft1TgCfm3Ek7m5zQvyQRV0Z2RcOu/FEjB80fbYIaA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(136003)(39860400002)(346002)(396003)(376002)(451199018)(6486002)(478600001)(316002)(2906002)(5660300002)(41300700001)(66946007)(44832011)(8936002)(4326008)(8676002)(66556008)(66476007)(6916009)(38100700002)(86362001)(82960400001)(9686003)(26005)(1076003)(186003)(6666004)(6512007)(6506007)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?g4HZKpvVzgnn9tJMkkWvy1vnYY7mcbnoyOsJY43P+beGCzGhfWCP1JnsghH2?= =?us-ascii?Q?3rfZyLJ5j8ynPKVPWmXYqioc/N7cXUrRTgsrbXXsJISJF3oiQima6k0Ibf0j?= =?us-ascii?Q?bbEr9sBWSOxe7AWXDKHnCUi1y3y52sIdRIWNo1IW6CFhyJVlXIpooMqiqR0r?= =?us-ascii?Q?qUVi4Pa02pU3k0LTB2bZm/YNLreAYoMZIRfEPNoAYGdoNZfo+tfStvPnBLbB?= =?us-ascii?Q?pAmsMY8nFDSxp4fA7SnYl+B8uCF2wzbyBKTKKrzTe32YOkvu8bJKnJr2l9NC?= =?us-ascii?Q?Ku+Y+O301FmREkzaHM8eujMp3Lxm6Qzf2/cRp+usUIB5+wVmnplyP69W181V?= =?us-ascii?Q?ebCUn07FtFHY3SBpoL0PZzmN71sV/E/NbkvzPq2JL6mE7KvFUdQQu7KVxBHP?= =?us-ascii?Q?eNa9Cs3Cb+te91Dk+DiQd5V80NA6w/s5o0CipCzYbxyS5L9qp+dsg+5rCM0e?= =?us-ascii?Q?AQPVwTfn55kPOrD1UfZAlE0Jns+aMtGvnRffoVFdsKWvoXpDdTOKi6OZInpJ?= =?us-ascii?Q?ECqB+UBDuzfRwoVYNk3AA0lAdp8j1lIPDS/kHNFfYl/t0lIlajfZShSPs1Es?= =?us-ascii?Q?TslBCEj3DQ0IeIIYpvwP+UHgdN27TIVtIA+eJtLr1hc2yJDrl27NnSexg79A?= =?us-ascii?Q?TgLkdTXNe/kheY5tzqt1AMVYqEVtaddstScMh+l875zhaJSL4GlWC9u6N3CA?= =?us-ascii?Q?DbFm+MNBiuM7wbEnRSzYgmGPj5cAgXnGoU0uDH21IsDKEKHxfUqV6rnktMo5?= =?us-ascii?Q?89O3HR0i3f+LqKw3Btfdgn3zJ5C8WiGI2up7QBG0tlkwNyYjHZgptoekpVkL?= =?us-ascii?Q?ry3IM+UWQaQJvbWioOQ1NmH85zJkF3Gibd5kOVBdjM+eGgaimwWwJHpu79qc?= =?us-ascii?Q?C51t7CvpFpZGI02IjEDIvi4MgCwa8o+6iO1+d+FZ/pc0jgF+t4EiYMuRAtJw?= =?us-ascii?Q?HZLF9jOEpITlXhOeENDm6By/WndYPdmRj/o445trqmtpEii085hQZO/VE4jI?= =?us-ascii?Q?9mD8mSr1L/JxvAsrfuQ7nqHmkCuHf6Of1hTxkag2amB6SHncQctXFG3jMxld?= =?us-ascii?Q?5QCS0eTeO1E+RSECeJ74+87/nAEtAF0GOqRVJ63CeOHLgYNCTmW/hhymkm6t?= =?us-ascii?Q?hx70Xu7s5DC1xu7wCt6h+IIvLgGzT0+ZEsj39ftmObzomQ7Nlasv4hgvaOpH?= =?us-ascii?Q?a1ZyJC0hoSw2kLdMF0eCJFeew/lS6h6oziOq1lSHsbq7zz+AUZfDYCm1nekU?= =?us-ascii?Q?aCtcznxH/FvuJB1eZ7Qao8lpNOXHIMzPJDTaW26H+yLe2QOYtYEXhUcjwjpv?= =?us-ascii?Q?yaRBO11ry6bA1ZdQp8tNsfmD0Vk+LJGsKWV7AVd7ZRWa1oD3VNPuz9mtcU7c?= =?us-ascii?Q?MJc9U+ZWIT+3/PjUo3QE3JI3gentBJVGSmezYIC6wKj0jhHvGdDQrkjwDmvU?= =?us-ascii?Q?6aYfx9wClzmFYt+BolDZ0NsgayYljwe2OXErLm/BiaH8DZaB5b046F7nszud?= =?us-ascii?Q?Aiz6sz2BQHZ7Ezz0AAKJWcvC0UfYvcIxFtq+ksU6xdlYIEwwjVdZ9zIyPNzJ?= =?us-ascii?Q?SIMGeKBVMzho5WTpJRSdMlua+us7VnKMQTKyGqFb4dgJSsW4mZC8VeG3o9I6?= =?us-ascii?Q?dA=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0847b6fe-acb7-4270-d0bf-08db292345b8 X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2023 09:12:47.5868 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: DgjqKRrOoZsMVQlE2LZSLeFcbzpge5yVO9DlEny283KJznsc6ZsmEZ8SXJkKoEuRlFKc/mwNTj0R2+AnppRH6lU4iYZV/PIt8IB/xJlGCNM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR10MB5312 X-TUID: yJd1eer6DGJW Am Mon, 20 Mar 2023 11:51:57 +0300 schrieb Uladzimir Bely : > In mail from Friday, 17 March 2023 11:53:58 +03 user Henning Schild > wrote: > > Am Fri, 17 Mar 2023 05:11:30 +0100 > > > > schrieb Uladzimir Bely : > > > When qemu machine boots, it may happen that consecutive SSH > > > connection fails right after the previous good one. So, we get a > > > situation when the command/script fails after we consider SSH is > > > ready. > > > > > > This patch improves detection of SSH server ready status by > > > making at least three good consecutive SSH pings. > > > > > > Example of debug output that shows the case: > > > > > > ``` > > > > > > | Waiting for SSH server ready... > > > | SSH ping result: 255, left: 300s # <== machine is booting > > > | SSH ping result: 255, left: 294s > > > | SSH ping result: 255, left: 288s > > > | SSH ping result: 255, left: 282s > > > | SSH ping result: 255, left: 276s > > > | SSH ping result: 255, left: 270s > > > | SSH ping result: 255, left: 264s > > > | SSH ping result: 255, left: 258s > > > | SSH ping result: 0, left: 253s # <== SSH server is up... > > > | SSH ping result: 0, left: 251s > > > | SSH ping result: 255, left: 250s # <== but one ping failed again > > > | SSH ping result: 0, left: 248s > > > | SSH ping result: 0, left: 245s > > > | SSH ping result: 0, left: 243s > > > | SSH server is ready > > > | `lsmod | grep example_module` returned 0 > > > > > > ``` > > > > > > Signed-off-by: Uladzimir Bely > > > --- > > > > > > testsuite/cibuilder.py | 12 ++++++++++-- > > > 1 file changed, 10 insertions(+), 2 deletions(-) > > > > > > diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py > > > index 9e84c3a3..4e568b8e 100755 > > > --- a/testsuite/cibuilder.py > > > +++ b/testsuite/cibuilder.py > > > > > > @@ -257,17 +257,25 @@ class CIBuilder(Test): > > > self.log.debug('Waiting for SSH server ready...') > > > > > > rc = None > > > > > > + goodcnt = 0 > > > > > > while time.time() < timeout: > > > if proc.poll() is not None: > > > self.log.error('Machine is not running') > > > return rc > > > > > > rc = self.exec_cmd('/bin/true', cmd_prefix) > > > > > > + time_left = timeout - time.time() > > > + self.log.debug('SSH ping result: %d, left: %.fs' % > > > (rc, time_left)) time.sleep(1) > > > > > > if rc == 0: > > > - self.log.debug('SSH server is ready') > > > - break > > > + goodcnt += 1 > > > + # Let 3 good SSH pings to make sure SSH > > > connection is stable > > > + if goodcnt >= 3: > > > + self.log.debug('SSH server is ready') > > > + break > > > + else: > > > + goodcnt = 0 > > > > This looks like an endless loop should ssh never come up. Not sure > > what would break that loop. In the worst case a test-timeout and > > everything being stuck because we might not execute these things in > > parallel. > > In case ssh never come up, we will exit after the timeout (default > value 600sec), goodcnt will always be 0 in this case. > > Using something more readable like "for i in range(3)" you've > suggested in the previous comment won't work because we need to reset > the counter and start again in case one of pings goes wrong. E.g, if > second ping fails, we need `goodcnt` changes like 0-0-0-1-0-1-2-3. > With "range" function it will act like 0-0-0-1-0-3. It is a nested loop, obviously the outer one needs to be changed a bit as well. But the inner can maybe be done with range. Henning > But you are right, code like "while time.time() < timeout and goodcnt > < 3" might be a bit more readable. I'll check it. > > > Henning > > > > > if rc != 0: > > > self.log.error('SSH server is not ready') > > > >