Details
-
Sub-task
-
Resolution: Out of scope
-
P2: Important
-
None
Description
It would be useful to execute benchmarks on more than one physical host for a given platform/branch combination. This could eliminate false positives arising from certain unintended/uncontrollable changes in performance characteristics in the test environment (disk fragmentation is one example). More precisely, we could make such a host correlation analysis influence the score of a performance regression (i.e. the more hosts that show the regression, the higher the score).