As far as I'm concerned, the variance on DYEBench should be no more than 5%.
e.g. Safari consistently reports the variance below 1% on many systems I've tested.
Since the variance of more than 5% indicates something went wrong,
we shouldn't even report the arithmetic mean.
Created attachment 231239 [details]
By the way, the "variance" I'm talking about here isn't really http://en.wikipedia.org/wiki/Variance.
I'm rejecting results with a confidence interval that is larger than 10% of the arithmetic mean.
Comment on attachment 231239 [details]
Per email discussion with Phil, we're going to increase the number of iterations from 5 to 20 instead.
Filed https://bugs.webkit.org/show_bug.cgi?id=132795 to track that.