From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7236574603632443392 X-Received: by 2002:a17:906:5a67:b0:965:62c3:7499 with SMTP id my39-20020a1709065a6700b0096562c37499mr5572761ejc.11.1684928268658; Wed, 24 May 2023 04:37:48 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aa7:c78a:0:b0:513:ede5:64d4 with SMTP id n10-20020aa7c78a000000b00513ede564d4ls5101797eds.0.-pod-prod-00-eu; Wed, 24 May 2023 04:37:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5XkYSKbeYdoV5o6uyKHOwQGkmKjmtC2xfqsG5ZWduvJsOws76gq9sMeAgFrlESMtReFth5 X-Received: by 2002:a05:6402:34cc:b0:506:72f8:eb10 with SMTP id w12-20020a05640234cc00b0050672f8eb10mr2294976edc.0.1684928267060; Wed, 24 May 2023 04:37:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1684928267; cv=pass; d=google.com; s=arc-20160816; b=OqP1+ZSwEgz1WOaMEyilAoxFIAOOQ7DOQN8daKyJbhm/BJqcAgI5RLUnkvzyzmxxto IQ2qQEkZTBAm7ysq8Ms7IC7Jvb95+0++oEr8qXwFfVLWuwsygqYEu5cVhiNPvfWxAh8g JG+Sy4xZtP+MuTzDjj1WZPGEb1RN8q/CMQlwqFJsOo7an55+x2gx223ef7Rp4PXVUAZg 7Wrq5MnmBZ2LNwm2kvZlpdiCdCv3k4UbRmjHTxfcDzbJVrkN01dTQezN01J3eBT2nPsm JIhT0Ci0qwW8V2u0CFu+b8/9o9931maPlY1uk9fQeaTOgE9nI1ljNdnAIwzXd3H4JPRd vtJg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:in-reply-to:from:references :to:content-language:subject:user-agent:date:message-id :dkim-signature; bh=bheOsSAmwlE0rc2AEPE+wvQoEvSq9AoBQ1veNBygjvE=; b=jt0PaFi+IBaTiKa6I0M6CKEE4ydQimIzZVDHzLcpdtTx/5QKkyjApFY8Js9yBhfg2S gbbWIGwEE3Q4FjEIF6+n+e/NDipPdBzpysJFGjbpSgsW3b3J6T6mXn6RogGtPhAvxAOn xNwQDIqcmLZKFAZqnWGvXWFtIJmc88LRhab9SGZb6XfoesMVSD54IwZ+eklTVudgpMhE xZIoMvM+KtifLMlg35g8DYLT+52XNa0AE2ECIYtrCraKY/llM7qtSuP+3NaGOLLGxBns lPWgWailQC0F0YppyPCqcwiIDBMU3jpm0hdOq4ypMs73LmRkbpvfW2CA5fysNeryHRbA ku0g== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=hRl0pFo8; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 2a01:111:f400:7e1a::621 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on20621.outbound.protection.outlook.com. [2a01:111:f400:7e1a::621]) by gmr-mx.google.com with ESMTPS id g1-20020a056402320100b0050bd0abf2b4si1043257eda.3.2023.05.24.04.37.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 May 2023 04:37:47 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 2a01:111:f400:7e1a::621 as permitted sender) client-ip=2a01:111:f400:7e1a::621; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=hRl0pFo8; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 2a01:111:f400:7e1a::621 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R6jnldq/23iCZDOJsmC1W58ozD12L7FnYCqT3bkmEr1H5JDAqQd3+Lz4kIyrMT9mnieQxjWYWii/reEUliy1PO0V6Pc43R4RnVuSzm4IF8wYEk6Lo/hHk9O35VMrNBRAlzUpQ7oUcOK6DH5jeBY3bui3nKZ8gyemHANVNSTRapajnWDP4xxvHbRwpo8CvZxig2jNWH+MCxmt2KyHKLEB+7/y15cKnjXBNeixa3tQP8Moa4FBbVWgHkpl88r3qMsh51cvpLIMYBHDQMl59OvYoARWbZa64yEtJ3eQaMuqvbReqrVgz5lqBaq5DATn4PbGbRUi/3KtkggAqgHZ5z9QXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bheOsSAmwlE0rc2AEPE+wvQoEvSq9AoBQ1veNBygjvE=; b=Km9S8Vu7kHP7Ukd3dRnefBMxjUpeMfTSjEJxQI7G6RK5AyU//YazwB9eqUGyRJ+iikirki4Cu7bqwwvoofCm+yvGjYMWmop0g6dj5ym+5kdZ0t5d4IA/6J/MU5SZJiJxVgNeQxX0+unx10PgAmxm1c0a8TWtX7NqJFu/75uykAf0Yo+GuSZmOWhn5ZSuHucih1neoRQ7F0NlFoELWp+kY3xmUJI4+27Rg7lIDjdNI0C4gBi+gZ+nasdDGHWN8f1cZtxU89jR7/G4h3u2ODqtOhG/+Jirudg9/uzF5wSzXRBJI1H1xHAr8pISL8zArEdWwY1NksxIPdrcLRWU6hu34g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bheOsSAmwlE0rc2AEPE+wvQoEvSq9AoBQ1veNBygjvE=; b=hRl0pFo8tk52n5ZLapNw9vZtR9Srydhug9HsAip6ZOxvP7UJK+MZcgM8RnGallEMF2ogfwbo1sXaxqMqFOisuBG+d+e4WyL6E7O0XnIRYD4yI0oatI2vp3uTKSwKfE1De+k4CivY91xeKzIz+oYIwXaL6SAEZjDdRaMktz1C+9H7GMeNRGwg8BV4uMoSEeyCtWVAWYqau1OnHDRfU2vQfYuOpctqZJ3cP4SD4M8S+GsBDpmB97JS/TCUjBrA7A0eLu0so7Th3T9W8SAjRqWMnbrvfYBhYv8E8lildcA8QUvAc2G3z/H6LsIIeCBY7OUvoPnOJ2S5gQdR4IB0we6bAg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:588::19) by AS2PR10MB7688.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:644::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.15; Wed, 24 May 2023 11:37:46 +0000 Received: from AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM ([fe80::d638:8333:21fb:5f18]) by AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM ([fe80::d638:8333:21fb:5f18%7]) with mapi id 15.20.6433.015; Wed, 24 May 2023 11:37:46 +0000 Message-ID: <2505b2c0-c1b9-534c-5ad6-ff3b1a625c06@siemens.com> Date: Wed, 24 May 2023 13:37:41 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH 0/4] Basic hardware testing support Content-Language: en-US To: Florian Bezdeka , "MOESSBAUER, Felix" , "ubely@ilbers.de" , "isar-users@googlegroups.com" References: <20230524024514.17042-1-ubely@ilbers.de> <06fd09e60c6af243f836bc06736b05218734e9d3.camel@siemens.com> <01510fd6fb6722d661b7d23e6e3ba4f89440498f.camel@siemens.com> From: Jan Kiszka In-Reply-To: <01510fd6fb6722d661b7d23e6e3ba4f89440498f.camel@siemens.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: FR0P281CA0215.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:ac::19) To AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:588::19) Return-Path: jan.kiszka@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS4PR10MB6181:EE_|AS2PR10MB7688:EE_ X-MS-Office365-Filtering-Correlation-Id: 7c06b5d3-4166-4cdd-88fb-08db5c4b4b4d X-LD-Processed: 38ae3bcd-9579-4fd4-adda-b42e1495d55a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: o5mGH8iecwu6TmghVefX3c+msNuxFibWdmjUy+e8RgicVk0hcOcuf9l4ExBzApj8bEgXpHV4CQm7kH4HZuccp9WNEWPF7bmTn4aRgdB19J320jRuApG2ECZpuyh56WhBki4ar6AjQrvDHmaRQWg97PdvHBPozcbAnkH6OZBDQDmGZ5W/XT84B1CwPDwmoG61b20p8krPEqDj5SP9M0Bnye2jOarJW8AEGZ2GiAOw82Mis/xFGCDA5ZiN0jw3+25ORUS1ijfQhgkPr63b9hNmEV5nHTc6KEfWD7S4U93zcsK/Zqa1elhYiV50/YS2JWP7M0gz15GrKzHuy1t48vVoeZjazKdD1msOByGNUaU95UI6bgoWpHfLbC0gpjpPgi7JlCF2WEXNB6Hi1UzEP83AWXQQ5QANeaLCJ5Dah682HWtP5bJXKmZsuI2QMLSX34MuQVDvVh+X1dGvuv1G49lnpHTUs3yCBDbetcd3a9bgTEUSWSWOSppuvZz4oiVMzyv+BkzejU7xCJr4CNDKadcLkY6xjU/8VqskTQnPTz2XRee4rVGxNvisLRWoOrYigCAC4WvDRM2/VBwWpHzKifDM/fEdhACfxQw3SxbJrPFEVfA5iqvMKL9ZXQ97S2e90hMYum3tN7n3R8APmMdetI757QDKw5hOTKlZvGNFo48pMRuPYuG/Af2SsBKjc63G/sRC X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230028)(4636009)(376002)(366004)(396003)(346002)(136003)(39860400002)(451199021)(8936002)(8676002)(44832011)(5660300002)(83380400001)(6512007)(6506007)(2616005)(53546011)(186003)(26005)(86362001)(31696002)(38100700002)(82960400001)(41300700001)(6486002)(966005)(6666004)(66946007)(66556008)(66476007)(316002)(36756003)(478600001)(110136005)(2906002)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGRBNGE1QUY1dDFlZ2JBdVhrMXpPdkhJUmIrRXJKRUFTQnhBbjU0NjdGanNk?= =?utf-8?B?MHRPOVROUWozejk5L2dwdG5IQXpWS0dnQUFmUDZGa0RwM1Urd3hUMVJoS0VG?= =?utf-8?B?WmdsQmhjMk9CNk1iYzBYR0EwZHNHVjh3bVlrSkRvelZjcDdyZ3hlZThrVnRE?= =?utf-8?B?WkNlMDErVkt3cnYyR2NtV0pIZ0IvbmIxN0xib2dENjJFVVdXSDdYZVppV1ZE?= =?utf-8?B?NTFPUmhqYVg4dlY0aDI1NmduNU1CcXludlpuN0Q0V3BXRXZyR2lnQ0NkUTJu?= =?utf-8?B?aEhRdDB1UEN0YUJtVU8vNGtmczNRTmVheXBUN1ZrS2dMTTV3elRBc3hiS1VJ?= =?utf-8?B?TnhzenBlalVPaWtPb0NnMG04M3JIZlE5eE84cWxqeVF3cFZOM04xSTE4cnhH?= =?utf-8?B?Q05iMFV3VFg3c29JL2tEa0VHWE1Cb2lhTzRmWmIxS0p0YjAxdHhIeU1jd1hE?= =?utf-8?B?N2JBVDdrTEZ0OHhiSnN6ZGhJUFZnUHBrYk1HUE8yLzVjWXV0MmlrelVONzZY?= =?utf-8?B?UnFFSWtVQk5XbDNTdy9wUmtyeWNXVElJa1V5VUE0a3E1NE5aSXEydmU5K0Nv?= =?utf-8?B?bXNwL3l6MGZFd3p1OG00aUlOcm5VYTd4R2hrSkpnaFZCWkNQbjhFOFQxdnA2?= =?utf-8?B?SWZRSFF0TThEa1o0RmRaMEd5eGtEMUI0QUdnRWJwYlZyTVJNMnNUZTNLakpz?= =?utf-8?B?UW9JS2xra2dyZFU0ejcxWEhzOTNBa3BzVDNhc2VaNytkQUkyYW05UVJ5QUZ2?= =?utf-8?B?RkdUd283NWlveXk5ckhDb3NQaTlTTkd6Tlgvb3FzSmFpWlNBTW5uZG5uc0hE?= =?utf-8?B?aE95K1R3OWZ2UTl6Z2NoT0hKTTNHS1lBNEdUSHN6SnhlS1R4NUNqUFZSOEVx?= =?utf-8?B?cmVkL1B6bkpZSzVtR29PZjhnYll0cWpiUEc5aUNPcVhBQmp4djBUeXM4d0pT?= =?utf-8?B?ZDF1LzRqY0h6VzZtTU40YmE5dlFrSlVIM1JRbEZDL1JBQnVoa0hHbHF0ZFc0?= =?utf-8?B?N0dvaEdIbGRqMWtNSEhuRVJwNzRUb0tpQzBVbFZFdHJsVVcrVEd0L1Fld3ox?= =?utf-8?B?K0tESnJvNTY3eU1mNXdoRFdObUxMVnJacVRKaGw2bEVpckt4L01zZ1ZoblFn?= =?utf-8?B?S05oMjd5T1c3K0pFT3p0cGozOFpZNDN0UjREUXF3eDdvb25iSTZoTlBSaC9k?= =?utf-8?B?dDZ4NGk3V20zUzBnQjdUSXlHSTJnYi9NYi8vVWdYTjFUL20zbVBHVXhCRDhX?= =?utf-8?B?UVdETWV2blNpZEVEVlN1M2NqNFJ1aWlvTlNwZlNhdjhJWXdjZ21PRUpJb3M1?= =?utf-8?B?VGJYQlk0VmxadlZxdjl6T080eUQ2aTdhZklJOWYzYWRuNGhkNmJpcUUrQXcx?= =?utf-8?B?TkRXMzRqa3kxeEp2WjFnZGM0TjNIRzRwcnFEQmp1OE9ZdCtQN2tJUElTS0J5?= =?utf-8?B?QUxQWDRyUmJQQko2Qmh3UE9vMU1lSWxBZHlvVGFIZXo3bjlJeVpHL3lUN3NF?= =?utf-8?B?YU5MeGNOV2FhVTVqcWw2WkYzdWs2ZVB0VWpEMFJlWFhaN0s4cE5TYkVJK0Ni?= =?utf-8?B?V1pITFRkWGxHVjBHQU1HMFB6V3B6QXdyeVJSdWtmeHNoOGpFNnRla2ptWlE1?= =?utf-8?B?bWRIa1FQTVNBaFZ5bE9tU2pxeDFibzNSVlVZV1pSZFRlY096MHJwdnQyckh4?= =?utf-8?B?YlBnNS9tZzZHMFdmd2E5dysyN2NHU1l2czdDb3I2YmR5cy9WSmo0U0N4eDRk?= =?utf-8?B?Y3F6MmtDYmh1LzI3KzlSS0NPQ084K0pZYXB5eTNvakVZNDNsM0p1NitRU243?= =?utf-8?B?V1Y3WUtnNTVRdEVva1pDTjBHMndMMmtob09tajYxT1lNTEY3RUowN2RXeUth?= =?utf-8?B?VVROdWVZZTJyY25XR0VicjRuellqcDU4VWszcGtWNElrUTdtM1NrajYveW1U?= =?utf-8?B?VkRRdUxWVFZld1VZNG1ucC94U1BHQzNGVVlxUTkxVXB2UlhSOFpZUEJDYVRJ?= =?utf-8?B?SlhqSkhwOTZHQ2YxTWdFcit5dmwzMU5QNzViaklDRjl3b0Vuc2hTUDRlRW5W?= =?utf-8?B?OWtjaGRzU0kwT3BLMUYxQmpEZUZlMXJ6Y0U3dVd4L2JXTGJtNXhIb1owMHRI?= =?utf-8?B?OHo0eEZsd0ZlTHNqRjlJWmY3NmxBTUlhMTBMeW1uRzlwaFlRK0hJZUF2NklP?= =?utf-8?B?U2c9PQ==?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7c06b5d3-4166-4cdd-88fb-08db5c4b4b4d X-MS-Exchange-CrossTenant-AuthSource: AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2023 11:37:46.0136 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CDZm1HUukgv/oO85azBRfUO6PQ5+Od2yg0iE68fG+UWu2DnDIS9AujOuKumTHc2be+ZTAf36ITebLVInxxrT6Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR10MB7688 X-TUID: 6MxqZZedf/rQ On 24.05.23 06:30, 'Florian Bezdeka' via isar-users wrote: > On Wed, 2023-05-24 at 03:06 +0000, 'MOESSBAUER, Felix' via isar-users > wrote: >> On Wed, 2023-05-24 at 04:45 +0200, Uladzimir Bely wrote: >>> Isar is currently able to run tests only on qemu targets over SSH. >>> >>> This patchset adds basic support of running tests on real hardware >>> targets in similar way. There are the following limitations: >>> >>> - User `ci` should be pre-configured (e.g., isar-ci-ssh-setup recipe >>> used when generating the image), similar to qemu testing; >>> - The image should be flashed on the board before running tests; >>> - The target should be turned on and accessible over SSH; >>> >>> Possible ways to improve this: >>> - Support turning on/off the hardware (via USB switches or similar); >>> - Automation of board software update. >> >> The flashing and powerctl features are implemented in MTDA [1]. We also >> use that in a lab for CI tests of ISAR images on hardware devices. > > I would say flashing is not necessary or at least no hard requirement. > We have LAVA (tool behind kernelci) setups deployed that implement NFS > based booting of ISAR generated images. If the purpose of the test is also validating the correctness of a wic run e.g., we do need to feed a real image in. We chould use MTDA's capability to emulate a USB stick and let the target boot from there (doing that for many targets by now for exactly that purpose). Jan > > One public available instance is lava.xenomai.org. Example test runs > can be found at [1] with the CI triggers coming from the xenomai-images > project [2]. > > Florian > > [1] https://lava.xenomai.org/scheduler/alljobs > [2] https://source.denx.de/Xenomai/xenomai-images > >> >> Felix >> >> [1] https://github.com/siemens/mtda >> >>> >>> Uladzimir Bely (4): >>>   cibuilder.py: Refactor vm_start related functions >>>   testsuite: Add an interface to run commands over ssh >>>   testsuite: Separate common part of kas-based test example >>>   testsute: Provide an example of hardware test >>> >>>  meta-isar/test/README.md                | 24 ++++++- >>>  meta-isar/test/common.sh                | 30 +++++++++ >>>  meta-isar/test/run_test.sh              | 33 ++------- >>>  meta-isar/test/run_test_hw.sh           | 10 +++ >>>  meta-isar/test/sample_kas_config_hw.yml | 32 +++++++++ >>>  meta-isar/test/sample_test.py           |  0 >>>  meta-isar/test/sample_test_hw.py        | 13 ++++ >>>  testsuite/cibuilder.py                  | 89 +++++++++++++++++------ >>> -- >>>  8 files changed, 174 insertions(+), 57 deletions(-) >>>  create mode 100755 meta-isar/test/common.sh >>>  create mode 100755 meta-isar/test/run_test_hw.sh >>>  create mode 100644 meta-isar/test/sample_kas_config_hw.yml >>>  mode change 100644 => 100755 meta-isar/test/sample_test.py >>>  create mode 100755 meta-isar/test/sample_test_hw.py >>> >>> -- >>> 2.20.1 >>> >> > -- Siemens AG, Technology Competence Center Embedded Linux