Bug 124496 - Simplify and reformat the output of performance tests inside test runners
Summary: Simplify and reformat the output of performance tests inside test runners
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryosuke Niwa
URL:
Keywords:
Depends on: 124495 124499
Blocks: 124623
  Show dependency treegraph
 
Reported: 2013-11-18 05:05 PST by Ryosuke Niwa
Modified: 2013-11-19 18:54 PST (History)
6 users (show)

See Also:


Attachments
Patch (22.28 KB, patch)
2013-11-18 06:51 PST, Ryosuke Niwa
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryosuke Niwa 2013-11-18 05:05:57 PST
We should dramatically streamline the raw output of performance tests in order to support subtests.
Comment 1 Ryosuke Niwa 2013-11-18 06:51:07 PST
Created attachment 217194 [details]
Patch
Comment 2 WebKit Commit Bot 2013-11-18 15:38:16 PST
Comment on attachment 217194 [details]
Patch

Clearing flags on attachment: 217194

Committed r159465: <http://trac.webkit.org/changeset/159465>
Comment 3 WebKit Commit Bot 2013-11-18 15:38:18 PST
All reviewed patches have been landed.  Closing bug.
Comment 4 Darin Adler 2013-11-19 09:17:21 PST
Comment on attachment 217194 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=217194&action=review

> PerformanceTests/ChangeLog:32
> +        getAttribute -> [1105, 1108, 1134, 1137, 1154]
> +        element.property -> [1634, 1655, 1685, 1696, 1723]
> +        setAttribute -> [646.3536463536464, 651, 651, 656.3436563436563, 658]
> +        element.property = value -> [934, 949, 963, 964, 974]
> +        element.expando = value -> [419, 419.5804195804196, 421.57842157842157, 425.57442557442556, 429]
> +        element.expando -> [501, 517, 519.4805194805194, 521.4785214785214, 525]
> +
> +            1: 117.40644785571585 runs/s
> +            2: 118.84720469666297 runs/s
> +            3: 119.80547640905021 runs/s
> +            4: 120.51886194758805 runs/s
> +            5: 121.51924380569295 runs/s
> +
> +        :Time -> [117.40644785571585, 118.84720469666297, 119.80547640905021, 120.51886194758805, 121.51924380569295] runs/s
> +            mean: 119.619446942942 runs/s
> +            median: 119.80547640905021 runs/s
> +            stdev: 1.5769040458730506 runs/s
> +            min: 117.40644785571585 runs/s
> +            max: 121.51924380569295 runs/s

This is showing numbers with precision that is not justified by the precision of our ability to measure. At some point we should fix to round so we don’t display so many meaningless digits.
Comment 5 Ryosuke Niwa 2013-11-19 18:54:07 PST
(In reply to comment #4)
> (From update of attachment 217194 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=217194&action=review
>
> This is showing numbers with precision that is not justified by the precision of our ability to measure. At some point we should fix to round so we don’t display so many meaningless digits.

That involves computing the confidence interval and computing the appropriate precision.  This is tracked by https://bugs.webkit.org/show_bug.cgi?id=100037.