From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7016230395466219520 X-Received: by 2002:a1c:2b04:: with SMTP id r4mr1388527wmr.48.1637167916856; Wed, 17 Nov 2021 08:51:56 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:f1c2:: with SMTP id z2ls4865wro.2.gmail; Wed, 17 Nov 2021 08:51:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJzu0Sj0B11tt+aZ5zJkfvUOzxnQMkIv7WNo3OlT/8xl8nC87qjA1z4HiUmxDG8oKXHc5dUn X-Received: by 2002:adf:ea0c:: with SMTP id q12mr21471793wrm.299.1637167915935; Wed, 17 Nov 2021 08:51:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637167915; cv=none; d=google.com; s=arc-20160816; b=YJN8BYL07OPA4hfwJa0mqn7lfhCjd9n+Mo02O4IHXTsEmWUqt/xRZ23W6qlpxYl02t zTh4b/lgf3waijWeTfJwnP/EqNVWj73JYiJRYLXYzJib2v2HcUBNjHHIXS/4u2L3ZpFo N9/VprIzFV/e1sTdeBMjxVgBQuMYPtJ4J8BGUMYwctLzJKpo7PqjsDNPcj5h2KFIE3Ns 3ZxANezomIU8hKLES1XVZtZPZw9/PO8SH0iEZITnPHs/R3L8rQfG6D6Lx37i5LDaf3zC OA07M/THXpOtyEa3QCyVxSybxohGk94m1ACJlKSzZStXpLjtlJU5sX6XLSP3CVivJyf5 7bPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:in-reply-to:mime-version :user-agent:date:message-id:cc:from:references:to:subject; bh=OTvYj9xSZ0LCtwCO1zqBXFkuLBFiU3qQ8wjm5djAO+Y=; b=mdEeEhGmq+3/PZffRNyFiETit6/XQEldN5G0rCzf2GuzKOP/hcQ1EDSjqccWpA1jvU 0JiSUUr1NUyhH58viNeSQEotdvEjfzqnpmuI5eXLTyyrjTgnidW+FAL3BYot3ekhccoM JFr1dCKbUTAJicWtubzYNdQNPQ2jQKl09g+5JG1OhTx+W5NsFmF8C02yiO5DrOyyoqta sS/S+LHPOdW6AnUSMHC97GHrpmR/LexvrvQ0aB97xfGlohayA7GfF/LohMmKAzkYzlQG qncCTYs8qv2BDcMEegWzSG221u46UbC5MzFmV5P8vcPxZvWBHmD0nqY0UeMwCWe7ABrx /8gQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 138si13932wme.0.2021.11.17.08.51.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 17 Nov 2021 08:51:55 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [192.168.67.164] (mm-55-56-214-37.mgts.dynamic.pppoe.byfly.by [37.214.56.55] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 1AHGpoNm012654 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 17 Nov 2021 17:51:54 +0100 Subject: Re: [PATCH v6 13/13] sstate: add test case To: "Schmidt, Adriaan" , "isar-users@googlegroups.com" References: <20211111064728.2375760-1-adriaan.schmidt@siemens.com> <20211111064728.2375760-14-adriaan.schmidt@siemens.com> From: Anton Mikanovich Cc: Baurzhan Ismagulov , Henning Schild Message-ID: Date: Wed, 17 Nov 2021 19:51:45 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: zTwRpeKpM8t+ 17.11.2021 11:21, Schmidt, Adriaan wrote: > Anton Mikanovich, 16. November 2021 17:47: >> 11.11.2021 09:47, Adriaan Schmidt wrote: >>> This also adds a parameter `sstate` to the build function, which is >>> only passed as `1` for the actual sstate test. None of the other tests >>> uses sstate caching. >>> >> Thanks for adding test case, but if you need to add it to fast/full in >> should be moved to the beginning of the build_test.py (after repro >> maybe) to not cleanup images needed for start_vm testing. Executing it >> at the end will also remove all the logs of common tests. >> Maybe we can not remove tmp but just set any empty SSTATE_DIR? > The whole testing of the sstate-cache is about building with a "warm" cache (populated SSTATE_DIR), but without any previous artifacts in tmp. Also, having logs from previous builds will break the check for executed tasks, which reads log.task_order files. > > One solution I see is to use a dedicated build dir for that test. This could also be a general option, to make it explicit which test cases depend on each other (use the same build dir) and which are independent (have a separate build dir for each test). That would mean to have the build dir not as an Avocado parameter, but have each test case set it individually. > >> Changing testsuite API should probably be moved to the separate patch. > If we're changing API, I would also propose to use keyword arguments instead of positional ones, and setting of defaults. This would make things much more readable, e.g., turning: > build_dir, bb_args = self.prep('Sstate', '', 0, 1, 1) > self.perform_build_test(image_target, 0, 1, None) > into: > build_dir, bb_args = self.prep('Sstate', sstate=True, debsrc_cache=True) > self.perform_build_test(image_target, sstate=True) > > > I'm happy to propose something, either before or (which would probably make Henning happy) after merging the sstate feature. > > Adriaan > Using separate build dirs was already in TODO for the next avocado improvements as requirement for test cases parallelization. If talking about testing API extension we can try to pass dict of key-value params as an argument. It will allow us to add any custom params later. Everything besides test case looks good, so we can merge the first 12 patches separately and move testing into separate thread. -- Anton Mikanovich Promwad Ltd. External service provider of ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn, Germany +49 (89) 122 67 24-0 Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov