Bug 76132 - Need a script to run tests in PerformanceTests
Summary: Need a script to run tests in PerformanceTests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryosuke Niwa
URL:
Keywords:
Depends on:
Blocks: 10266
  Show dependency treegraph
 
Reported: 2012-01-11 17:15 PST by Ryosuke Niwa
Modified: 2012-01-12 14:08 PST (History)
7 users (show)

See Also:


Attachments
work in progress (6.96 KB, patch)
2012-01-11 17:34 PST, Ryosuke Niwa
no flags Details | Formatted Diff | Diff
Patch (16.96 KB, patch)
2012-01-12 02:07 PST, Ryosuke Niwa
no flags Details | Formatted Diff | Diff
Added a change log entry (17.31 KB, patch)
2012-01-12 02:11 PST, Ryosuke Niwa
abarth: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryosuke Niwa 2012-01-11 17:15:01 PST
We already have perftestsrunner for inspector tests per http://trac.webkit.org/changeset/101618. We should extend this script to be able to run other performance tests.
Comment 1 Ryosuke Niwa 2012-01-11 17:33:16 PST
The change log entry in http://trac.webkit.org/changeset/101618 seems to imply this script is meant to be used by some chromium perf bots. Did that happen already or are we still in the process of improving the script?
Comment 2 Ryosuke Niwa 2012-01-11 17:34:30 PST
Created attachment 122142 [details]
work in progress

With this patch, the script output will look like:

rniwa-macpro:webkit4 rniwa$ Tools/Scripts/run-perf-tests Bindings
Running Bindings/event-target-wrapper.html (1 of 1)
RESULT 1057.6
Comment 3 Adam Barth 2012-01-11 17:38:35 PST
Comment on attachment 122142 [details]
work in progress

View in context: https://bugs.webkit.org/attachment.cgi?id=122142&action=review

> Tools/Scripts/run-inspector-perf-tests.py:41
> -    sys.exit(PerfTestsRunner('inspector').run())
> +    sys.exit(PerfTestsRunner('inspector').run(inspector_tests=True))

Should the inspector have a subclass of PerfTestsRunner?
Comment 4 Ryosuke Niwa 2012-01-11 18:04:37 PST
Comment on attachment 122142 [details]
work in progress

View in context: https://bugs.webkit.org/attachment.cgi?id=122142&action=review

>> Tools/Scripts/run-inspector-perf-tests.py:41
>> +    sys.exit(PerfTestsRunner('inspector').run(inspector_tests=True))
> 
> Should the inspector have a subclass of PerfTestsRunner?

Oops, this change is a left over. Ideally, we should be able to run all perf tests and get the output in the same format so probably not.
Comment 5 Ryosuke Niwa 2012-01-12 02:07:54 PST
Created attachment 122192 [details]
Patch
Comment 6 Ryosuke Niwa 2012-01-12 02:11:41 PST
Created attachment 122193 [details]
Added a change log entry
Comment 7 Adam Barth 2012-01-12 11:12:03 PST
Comment on attachment 122193 [details]
Added a change log entry

What is a Chromium-style test?
Comment 8 Ryosuke Niwa 2012-01-12 11:14:44 PST
(In reply to comment #7)
> (From update of attachment 122193 [details])
> What is a Chromium-style test?

It's a test outputs the result that Chromium perf bots can parse natively.
Comment 9 Adam Barth 2012-01-12 13:41:56 PST
> It's a test outputs the result that Chromium perf bots can parse natively.

Is it a good format?  Should we change out PerformanceTests to match?
Comment 10 Adam Barth 2012-01-12 13:42:59 PST
Comment on attachment 122193 [details]
Added a change log entry

This looks ok.  I wish we didn't have two different formats to parse, but maybe that's not super important.
Comment 11 Ryosuke Niwa 2012-01-12 14:03:07 PST
(In reply to comment #9)
> > It's a test outputs the result that Chromium perf bots can parse natively.
> 
> Is it a good format?  Should we change out PerformanceTests to match?

No, it's actually hideous looks like:
RESULT Bindings: event-target-wrapper= 1033.3 ms

RESULT panel-switch: show-panel-elements= 10 ms
RESULT panel-switch: show-panel-resources= 13 ms
RESULT panel-switch: show-panel-network= 38 ms
RESULT panel-switch: show-panel-scripts= 20 ms
RESULT panel-switch: show-panel-timeline= 12 ms
RESULT panel-switch: show-panel-profiles= 10 ms
RESULT panel-switch: show-panel-audits= 7 ms
RESULT panel-switch: show-panel-console= 13 ms
RESULT heap-delta: show-panel= 0 kB
Comment 12 Ryosuke Niwa 2012-01-12 14:08:47 PST
Committed r104856: <http://trac.webkit.org/changeset/104856>