Bug 76132

Summary: Need a script to run tests in PerformanceTests
Product: WebKit Reporter: Ryosuke Niwa <rniwa>
Component: Tools / TestsAssignee: Ryosuke Niwa <rniwa>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, eric, loislo, morrita, ojan, tony, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 10266    
Attachments:
Description Flags
work in progress
none
Patch
none
Added a change log entry abarth: review+

Description Ryosuke Niwa 2012-01-11 17:15:01 PST
We already have perftestsrunner for inspector tests per http://trac.webkit.org/changeset/101618. We should extend this script to be able to run other performance tests.
Comment 1 Ryosuke Niwa 2012-01-11 17:33:16 PST
The change log entry in http://trac.webkit.org/changeset/101618 seems to imply this script is meant to be used by some chromium perf bots. Did that happen already or are we still in the process of improving the script?
Comment 2 Ryosuke Niwa 2012-01-11 17:34:30 PST
Created attachment 122142 [details]
work in progress

With this patch, the script output will look like:

rniwa-macpro:webkit4 rniwa$ Tools/Scripts/run-perf-tests Bindings
Running Bindings/event-target-wrapper.html (1 of 1)
RESULT 1057.6
Comment 3 Adam Barth 2012-01-11 17:38:35 PST
Comment on attachment 122142 [details]
work in progress

View in context: https://bugs.webkit.org/attachment.cgi?id=122142&action=review

> Tools/Scripts/run-inspector-perf-tests.py:41
> -    sys.exit(PerfTestsRunner('inspector').run())
> +    sys.exit(PerfTestsRunner('inspector').run(inspector_tests=True))

Should the inspector have a subclass of PerfTestsRunner?
Comment 4 Ryosuke Niwa 2012-01-11 18:04:37 PST
Comment on attachment 122142 [details]
work in progress

View in context: https://bugs.webkit.org/attachment.cgi?id=122142&action=review

>> Tools/Scripts/run-inspector-perf-tests.py:41
>> +    sys.exit(PerfTestsRunner('inspector').run(inspector_tests=True))
> 
> Should the inspector have a subclass of PerfTestsRunner?

Oops, this change is a left over. Ideally, we should be able to run all perf tests and get the output in the same format so probably not.
Comment 5 Ryosuke Niwa 2012-01-12 02:07:54 PST
Created attachment 122192 [details]
Patch
Comment 6 Ryosuke Niwa 2012-01-12 02:11:41 PST
Created attachment 122193 [details]
Added a change log entry
Comment 7 Adam Barth 2012-01-12 11:12:03 PST
Comment on attachment 122193 [details]
Added a change log entry

What is a Chromium-style test?
Comment 8 Ryosuke Niwa 2012-01-12 11:14:44 PST
(In reply to comment #7)
> (From update of attachment 122193 [details])
> What is a Chromium-style test?

It's a test outputs the result that Chromium perf bots can parse natively.
Comment 9 Adam Barth 2012-01-12 13:41:56 PST
> It's a test outputs the result that Chromium perf bots can parse natively.

Is it a good format?  Should we change out PerformanceTests to match?
Comment 10 Adam Barth 2012-01-12 13:42:59 PST
Comment on attachment 122193 [details]
Added a change log entry

This looks ok.  I wish we didn't have two different formats to parse, but maybe that's not super important.
Comment 11 Ryosuke Niwa 2012-01-12 14:03:07 PST
(In reply to comment #9)
> > It's a test outputs the result that Chromium perf bots can parse natively.
> 
> Is it a good format?  Should we change out PerformanceTests to match?

No, it's actually hideous looks like:
RESULT Bindings: event-target-wrapper= 1033.3 ms

RESULT panel-switch: show-panel-elements= 10 ms
RESULT panel-switch: show-panel-resources= 13 ms
RESULT panel-switch: show-panel-network= 38 ms
RESULT panel-switch: show-panel-scripts= 20 ms
RESULT panel-switch: show-panel-timeline= 12 ms
RESULT panel-switch: show-panel-profiles= 10 ms
RESULT panel-switch: show-panel-audits= 7 ms
RESULT panel-switch: show-panel-console= 13 ms
RESULT heap-delta: show-panel= 0 kB
Comment 12 Ryosuke Niwa 2012-01-12 14:08:47 PST
Committed r104856: <http://trac.webkit.org/changeset/104856>