Details
-
User Story
-
Resolution: Out of scope
-
P3: Somewhat important
-
None
-
None
-
None
Description
We should support the running of DLL swap style BC autotests in the (Qt/Mobility/etc) quality gates.
This form of BC testing is like this:
- compile some tests against version X of a product.
- run them and keep the results.
- run them again, against version >X of the product.
- compare the results from version X and >X. Should be identical, if >X is backwards binary compatible with X (which is our "promise" for most of our products)
The main thing needed from our infrastructure to support this, is the logic for installing multiple versions of the binaries for some product alongside each other and forcing the tests to use them (i.e. DLL swapping).
This task is only about the infrastructure support for this kind of test, and not providing the tests themselves. The tests may come from various sources. There are tests for Qt at http://gitorious.org/bc/ and Mobility at http://gitorious.org/bc-mobility/ which we could use.