From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7024137565936549888 X-Received: by 2002:a2e:a6a0:: with SMTP id q32mr4446593lje.344.1635445320753; Thu, 28 Oct 2021 11:22:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:c74d:: with SMTP id x74ls2864420lff.3.gmail; Thu, 28 Oct 2021 11:21:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzUrXJHhCvHLNPJO1oIW5zxeNc7xY02YaWUDyVFfNkykdMPGas7eykL+EK944xsIvO6K+LN X-Received: by 2002:ac2:4312:: with SMTP id l18mr5545081lfh.602.1635445319653; Thu, 28 Oct 2021 11:21:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635445319; cv=none; d=google.com; s=arc-20160816; b=SAVoEbRorufsNJXTo8gjAu13B9qgYyo+L87raStwtgqD2BB6mUp/c8WYFbyRw/U4Nl WAXEOPaXxtP4R0P2dxfWhQ4ifFk2hbL6uzjRy9b+pl8Q4uLHgVDNd3ui55dxfmfmK37h 6OhEN6GbqdDLAlo+cYO57y80jCI8ZhKyJPLs1RTlt4uuk7xv7tZ+UhwDBetXzPLulUBT ItJcanTwdVsEl5T+WQVVHp3w+L5u5BYCjtdPXqd1WBRrnu8Z4UsXPOvz0k2SJ0FI3e0Y mVNl41ZdCtpN17F9Ir4FAQoj6I4LUHKoTSqkjsA/GwbcP1CeqtoH6B0f3zH8jOKfAywY IpqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=tAovvMNjrS0Ka47bXYCkf4qec8Ku2BDT8R0uNH3WcSw=; b=hg23zsbOtfbuoZyK3NZM/oXXupstkRXWIQZw0ovTg8e77Sq0n6y+1PjuOT5n0JFkb7 ts+3rXBPFI4ar8ej30RGnyC6MVBL7zAK3MJeyXtBmmtuUuKSRVHyzrif9JoS+B9aF5Ma rh7K/ndvTJ8icd+HluTYEMr8wA4aPsAUhGhsedHDghKGv3RMQMY96mDAu/6RV0Jb/Wvc dvf+26auYp36hjwtqPrADXLmTJDbeVZczGSx7dRlrF6h4ejoKGJj12zEgNWj3kEv3LB0 YJ+vh1i88NQPt9R4TZlJDxPtvr3FUhBw6iugtSi2IsKLtGNCDKNT8eFwsglKIx1PmSnD PC1A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id v203si198173lfa.13.2021.10.28.11.21.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Oct 2021 11:21:59 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 19SILwAv031615 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 28 Oct 2021 20:21:59 +0200 Received: from md1za8fc.ad001.siemens.net ([167.87.32.106]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 19SILwRZ030908; Thu, 28 Oct 2021 20:21:58 +0200 Date: Thu, 28 Oct 2021 20:21:57 +0200 From: Henning Schild To: Anton Mikanovich Cc: Jan Kiszka , isar-users@googlegroups.com, "Schmidt, Adriaan" Subject: Re: [PATCH] CI: Add sstate-cache testcase Message-ID: <20211028202157.1ef109e5@md1za8fc.ad001.siemens.net> In-Reply-To: <1822c3ac-88d1-32cc-8886-c83d4746fe3a@ilbers.de> References: <20211028152056.244471-1-amikan@ilbers.de> <1822c3ac-88d1-32cc-8886-c83d4746fe3a@ilbers.de> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: fcT0OZOYz588 Am Thu, 28 Oct 2021 20:22:58 +0300 schrieb Anton Mikanovich : > 28.10.2021 18:34, Jan Kiszka wrote: > > On 28.10.21 17:20, Anton Mikanovich wrote: > >> Test rebuild time improve after cleanup to be sure sstate-cache > >> works. > >> > >> Signed-off-by: Anton Mikanovich > >> --- > >> testsuite/build_test/build_test.py | 40 > >> ++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 > >> deletions(-) > >> > >> diff --git a/testsuite/build_test/build_test.py > >> b/testsuite/build_test/build_test.py index d39c10c0..244f6fc0 > >> 100644 --- a/testsuite/build_test/build_test.py > >> +++ b/testsuite/build_test/build_test.py > >> @@ -1,9 +1,9 @@ > >> #!/usr/bin/env python3 > >> > >> -import os > >> +import os, time > >> > >> from avocado import skipUnless > >> -from avocado.utils import path > >> +from avocado.utils import path, process > >> from cibase import CIBaseTest > >> > >> UMOCI_AVAILABLE = True > >> @@ -206,3 +206,39 @@ class ContainerSdkTest(CIBaseTest): > >> targets = ['mc:container-amd64-stretch:isar-image-base'] > >> > >> self.perform_container_test(targets, 'do_populate_sdk') > >> + > >> +class SstateTest(CIBaseTest): > >> + > >> + """ > >> + Test rebuild speed improve with sstate-cache > >> + > >> + :avocado: tags=sstate > >> + """ > >> + def test_sstate_rebuild(self): > >> + speedup_k = 2.0 > >> + > >> + targets = [ > >> + 'mc:qemuamd64-bullseye:isar-image-base' > >> + ] > >> + > >> + # Cleanup everything before build > >> + build_dir = self.params.get('build_dir', > >> + default=os.path.dirname(__file__) + > >> '/../../build') > >> + process.run('rm -rf ' + build_dir + '/sstate-cache', > >> sudo=True) > >> + self.deletetmp(build_dir) > >> + > >> + start = time.time() > >> + self.perform_build_test(targets, 1, None) > >> + first_time = time.time() - start > >> + self.log.info('Non-cached build:' + str(round(first_time, > >> 2))) + > >> + # Cleanup everything but cache files > >> + self.deletetmp(build_dir) > >> + > >> + start = time.time() > >> + self.perform_build_test(targets, 1, None) > >> + second_time = time.time() - start > >> + self.log.info('Cached build:' + str(round(second_time, > >> 2))) + > >> + if first_time / second_time < speedup_k: > >> + self.fail('No speedup after rebuild with > >> sstate-cache') > > Is there no better way than measuring time to test if sstate was in > > place? Maybe some hit/miss statistics? > > > > Jan > > > I've tried to find some Avocado API to access final testcase > statistics from within other testcase but didn't find so far. > Any advice is welcome. sstate hit rates are in the bitbake logs, i hope avocado can open artifacts But this is infra ... we will see when it fails and do not have to test that. The interesting problems will happen once you build version+5 on a cache from version. The only known limitiation there is that your base rootfss must not be too old. Because your local apt-database will start containing package versions that might not be available to install anymore. So you have to find an eviction strategy. But that is very project specific. We currently have two strategies in place on two projects happily working with this since weeks. 1. nightly clean + full build 2. "tmpreaper"-style age-greater 3 days Third one you build against a frozen mirror and never clear, or very infrequent. Plus this problem is well known to isar guys, you just have to sometimes do a full build or "bitbake -C task recipe". It just now moves to runners ... but only if those are specially configured to offer persistent storage. Not sure how jenkins deals with this per default, and what sort of configuration the ilbers jenkins might need. Henning