Details
-
User Story
-
Resolution: Done
-
Not Evaluated
-
None
-
None
-
None
Description
Autotests should be run in parallel where possible. Currently we have test machines with 4 or more CPUs wasting time running autotests one by one.
Unfortunately it is not possible to run all the autotests in parallel, because there are many system tests in Qt which expect exclusive access to some resource. The most common example is GUI tests which expect to be able to give their window focus. Therefore the autotests should be able to be flagged on a per-testcase basis as being parallel-safe or not. In fact, many tests already have a CONFIG+=parallel_test, but nothing in the build system or CI system cares about this.
Suggested implementation:
- make the build system care about CONFIG+=parallel_test
- a `make -j4 check' should run up to 4 parallel_test testcases at a time, but should run non-parallel tests one at a time
- do something intelligent with the output of tests running in parallel
- if nothing special is done, their output will be interleaved and very difficult to read
- validate the current usage of CONFIG+=parallel_test, as it may have been copy-pasted to various places without understanding.