Re: [PATCH] CI: Add sstate-cache testcase

[Anton Mikanovich <amikan@ilbers.de>]

28.10.2021 18:34, Jan Kiszka wrote:
> On 28.10.21 17:20, Anton Mikanovich wrote:
>> Test rebuild time improve after cleanup to be sure sstate-cache works.
>>
>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>> ---
>>   testsuite/build_test/build_test.py | 40 ++++++++++++++++++++++++++++--
>>   1 file changed, 38 insertions(+), 2 deletions(-)
>>
>> diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py
>> index d39c10c0..244f6fc0 100644
>> --- a/testsuite/build_test/build_test.py
>> +++ b/testsuite/build_test/build_test.py
>> @@ -1,9 +1,9 @@
>>   #!/usr/bin/env python3
>>   
>> -import os
>> +import os, time
>>   
>>   from avocado import skipUnless
>> -from avocado.utils import path
>> +from avocado.utils import path, process
>>   from cibase import CIBaseTest
>>   
>>   UMOCI_AVAILABLE = True
>> @@ -206,3 +206,39 @@ class ContainerSdkTest(CIBaseTest):
>>           targets = ['mc:container-amd64-stretch:isar-image-base']
>>   
>>           self.perform_container_test(targets, 'do_populate_sdk')
>> +
>> +class SstateTest(CIBaseTest):
>> +
>> +    """
>> +    Test rebuild speed improve with sstate-cache
>> +
>> +    :avocado: tags=sstate
>> +    """
>> +    def test_sstate_rebuild(self):
>> +        speedup_k = 2.0
>> +
>> +        targets = [
>> +            'mc:qemuamd64-bullseye:isar-image-base'
>> +                  ]
>> +
>> +        # Cleanup everything before build
>> +        build_dir = self.params.get('build_dir',
>> +                    default=os.path.dirname(__file__) + '/../../build')
>> +        process.run('rm -rf ' + build_dir + '/sstate-cache', sudo=True)
>> +        self.deletetmp(build_dir)
>> +
>> +        start = time.time()
>> +        self.perform_build_test(targets, 1, None)
>> +        first_time = time.time() - start
>> +        self.log.info('Non-cached build:' + str(round(first_time, 2)))
>> +
>> +        # Cleanup everything but cache files
>> +        self.deletetmp(build_dir)
>> +
>> +        start = time.time()
>> +        self.perform_build_test(targets, 1, None)
>> +        second_time = time.time() - start
>> +        self.log.info('Cached build:' + str(round(second_time, 2)))
>> +
>> +        if first_time / second_time < speedup_k:
>> +            self.fail('No speedup after rebuild with sstate-cache')
>>
> Is there no better way than measuring time to test if sstate was in
> place? Maybe some hit/miss statistics?
>
> Jan
>
I've tried to find some Avocado API to access final testcase statistics 
from within other testcase but didn't find so far.
Any advice is welcome.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov