RE: [PATCH v6 13/13] sstate: add test case

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

Anton Mikanovich, 17. November 2021 17:52:
> 17.11.2021 11:21, Schmidt, Adriaan wrote:
> > Anton Mikanovich, 16. November 2021 17:47:
> >> 11.11.2021 09:47, Adriaan Schmidt wrote:
> >>> This also adds a parameter `sstate` to the build function, which is
> >>> only passed as `1` for the actual sstate test. None of the other tests
> >>> uses sstate caching.
> >>>
> >> Thanks for adding test case, but if you need to add it to fast/full in
> >> should be moved to the beginning of the build_test.py (after repro
> >> maybe) to not cleanup images needed for start_vm testing. Executing it
> >> at the end will also remove all the logs of common tests.
> >> Maybe we can not remove tmp but just set any empty SSTATE_DIR?
> > The whole testing of the sstate-cache is about building with a "warm" cache
> (populated SSTATE_DIR), but without any previous artifacts in tmp. Also,
> having logs from previous builds will break the check for executed tasks,
> which reads log.task_order files.
> >
> > One solution I see is to use a dedicated build dir for that test. This
> could also be a general option, to make it explicit which test cases depend
> on each other (use the same build dir) and which are independent (have a
> separate build dir for each test). That would mean to have the build dir not
> as an Avocado parameter, but have each test case set it individually.
> >
> >> Changing testsuite API should probably be moved to the separate patch.
> > If we're changing API, I would also propose to use keyword arguments
> instead of positional ones, and setting of defaults. This would make things
> much more readable, e.g., turning:
> > build_dir, bb_args = self.prep('Sstate', '', 0, 1, 1)
> > self.perform_build_test(image_target, 0, 1, None)
> > into:
> > build_dir, bb_args = self.prep('Sstate', sstate=True, debsrc_cache=True)
> > self.perform_build_test(image_target, sstate=True)
> >
> >
> > I'm happy to propose something, either before or (which would probably make
> Henning happy) after merging the sstate feature.
> >
> > Adriaan
> >
> Using separate build dirs was already in TODO for the next avocado
> improvements as requirement for test cases parallelization.
> If talking about testing API extension we can try to pass dict of
> key-value params as an argument. It will allow us to add any custom
> params later.

I have something that addresses parameter-passing and almost runs.
Will post a refactoring proposal soon.

> Everything besides test case looks good, so we can merge the first 12
> patches separately and move testing into separate thread.

I agree. But only merge patches 1..10, and leave everything test-related for another thread.

Adriaan