Bug 124496 – Simplify and reformat the output of performance tests inside test runners

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Ryosuke Niwa]

We should dramatically streamline the raw output of performance tests in order to support subtests.

[Ryosuke Niwa]

Created attachment 217194 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 217194 [details]
Patch

Clearing flags on attachment: 217194

Committed r159465: <http://trac.webkit.org/changeset/159465>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Darin Adler]

Comment on attachment 217194 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=217194&action=review

> PerformanceTests/ChangeLog:32
> +        getAttribute -> [1105, 1108, 1134, 1137, 1154]
> +        element.property -> [1634, 1655, 1685, 1696, 1723]
> +        setAttribute -> [646.3536463536464, 651, 651, 656.3436563436563, 658]
> +        element.property = value -> [934, 949, 963, 964, 974]
> +        element.expando = value -> [419, 419.5804195804196, 421.57842157842157, 425.57442557442556, 429]
> +        element.expando -> [501, 517, 519.4805194805194, 521.4785214785214, 525]
> +
> +            1: 117.40644785571585 runs/s
> +            2: 118.84720469666297 runs/s
> +            3: 119.80547640905021 runs/s
> +            4: 120.51886194758805 runs/s
> +            5: 121.51924380569295 runs/s
> +
> +        :Time -> [117.40644785571585, 118.84720469666297, 119.80547640905021, 120.51886194758805, 121.51924380569295] runs/s
> +            mean: 119.619446942942 runs/s
> +            median: 119.80547640905021 runs/s
> +            stdev: 1.5769040458730506 runs/s
> +            min: 117.40644785571585 runs/s
> +            max: 121.51924380569295 runs/s

This is showing numbers with precision that is not justified by the precision of our ability to measure. At some point we should fix to round so we don’t display so many meaningless digits.

[Ryosuke Niwa]

(In reply to comment #4)
> (From update of attachment 217194 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=217194&action=review
>
> This is showing numbers with precision that is not justified by the precision of our ability to measure. At some point we should fix to round so we don’t display so many meaningless digits.

That involves computing the confidence interval and computing the appropriate precision.  This is tracked by https://bugs.webkit.org/show_bug.cgi?id=100037.