Re: [PATCH 0/4] Basic hardware testing support

[Jan Kiszka <jan.kiszka@siemens.com>]

On 24.05.23 06:30, 'Florian Bezdeka' via isar-users wrote:
> On Wed, 2023-05-24 at 03:06 +0000, 'MOESSBAUER, Felix' via isar-users
> wrote:
>> On Wed, 2023-05-24 at 04:45 +0200, Uladzimir Bely wrote:
>>> Isar is currently able to run tests only on qemu targets over SSH.
>>>
>>> This patchset adds basic support of running tests on real hardware
>>> targets in similar way. There are the following limitations:
>>>
>>> - User `ci` should be pre-configured (e.g., isar-ci-ssh-setup recipe
>>> used when generating the image), similar to qemu testing;
>>> - The image should be flashed on the board before running tests;
>>> - The target should be turned on and accessible over SSH;
>>>
>>> Possible ways to improve this:
>>> - Support turning on/off the hardware (via USB switches or similar);
>>> - Automation of board software update.
>>
>> The flashing and powerctl features are implemented in MTDA [1]. We also
>> use that in a lab for CI tests of ISAR images on hardware devices.
> 
> I would say flashing is not necessary or at least no hard requirement.
> We have LAVA (tool behind kernelci) setups deployed that implement NFS
> based booting of ISAR generated images.

If the purpose of the test is also validating the correctness of a wic
run e.g., we do need to feed a real image in. We chould use MTDA's
capability to emulate a USB stick and let the target boot from there
(doing that for many targets by now for exactly that purpose).

Jan

> 
> One public available instance is lava.xenomai.org. Example test runs
> can be found at [1] with the CI triggers coming from the xenomai-images
> project [2].
> 
> Florian
> 
> [1] https://lava.xenomai.org/scheduler/alljobs
> [2] https://source.denx.de/Xenomai/xenomai-images
> 
>>
>> Felix
>>
>> [1] https://github.com/siemens/mtda
>>
>>>
>>> Uladzimir Bely (4):
>>>   cibuilder.py: Refactor vm_start related functions
>>>   testsuite: Add an interface to run commands over ssh
>>>   testsuite: Separate common part of kas-based test example
>>>   testsute: Provide an example of hardware test
>>>
>>>  meta-isar/test/README.md                | 24 ++++++-
>>>  meta-isar/test/common.sh                | 30 +++++++++
>>>  meta-isar/test/run_test.sh              | 33 ++-------
>>>  meta-isar/test/run_test_hw.sh           | 10 +++
>>>  meta-isar/test/sample_kas_config_hw.yml | 32 +++++++++
>>>  meta-isar/test/sample_test.py           |  0
>>>  meta-isar/test/sample_test_hw.py        | 13 ++++
>>>  testsuite/cibuilder.py                  | 89 +++++++++++++++++------
>>> --
>>>  8 files changed, 174 insertions(+), 57 deletions(-)
>>>  create mode 100755 meta-isar/test/common.sh
>>>  create mode 100755 meta-isar/test/run_test_hw.sh
>>>  create mode 100644 meta-isar/test/sample_kas_config_hw.yml
>>>  mode change 100644 => 100755 meta-isar/test/sample_test.py
>>>  create mode 100755 meta-isar/test/sample_test_hw.py
>>>
>>> -- 
>>> 2.20.1
>>>
>>
> 

-- 
Siemens AG, Technology
Competence Center Embedded Linux