Bug 76132 – Need a script to run tests in PerformanceTests

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Ryosuke Niwa]

We already have perftestsrunner for inspector tests per http://trac.webkit.org/changeset/101618. We should extend this script to be able to run other performance tests.

[Ryosuke Niwa]

The change log entry in http://trac.webkit.org/changeset/101618 seems to imply this script is meant to be used by some chromium perf bots. Did that happen already or are we still in the process of improving the script?

[Ryosuke Niwa]

Created attachment 122142 [details]
work in progress

With this patch, the script output will look like:

rniwa-macpro:webkit4 rniwa$ Tools/Scripts/run-perf-tests Bindings
Running Bindings/event-target-wrapper.html (1 of 1)
RESULT 1057.6

[Adam Barth]

Comment on attachment 122142 [details]
work in progress

View in context: https://bugs.webkit.org/attachment.cgi?id=122142&action=review

> Tools/Scripts/run-inspector-perf-tests.py:41
> -    sys.exit(PerfTestsRunner('inspector').run())
> +    sys.exit(PerfTestsRunner('inspector').run(inspector_tests=True))

Should the inspector have a subclass of PerfTestsRunner?

[Ryosuke Niwa]

Comment on attachment 122142 [details]
work in progress

View in context: https://bugs.webkit.org/attachment.cgi?id=122142&action=review

>> Tools/Scripts/run-inspector-perf-tests.py:41
>> +    sys.exit(PerfTestsRunner('inspector').run(inspector_tests=True))
> 
> Should the inspector have a subclass of PerfTestsRunner?

Oops, this change is a left over. Ideally, we should be able to run all perf tests and get the output in the same format so probably not.

[Ryosuke Niwa]

Created attachment 122192 [details]
Patch

[Ryosuke Niwa]

Created attachment 122193 [details]
Added a change log entry

[Adam Barth]

Comment on attachment 122193 [details]
Added a change log entry

What is a Chromium-style test?

[Ryosuke Niwa]

(In reply to comment #7)
> (From update of attachment 122193 [details])
> What is a Chromium-style test?

It's a test outputs the result that Chromium perf bots can parse natively.

[Adam Barth]

> It's a test outputs the result that Chromium perf bots can parse natively.

Is it a good format?  Should we change out PerformanceTests to match?

[Adam Barth]

Comment on attachment 122193 [details]
Added a change log entry

This looks ok.  I wish we didn't have two different formats to parse, but maybe that's not super important.

[Ryosuke Niwa]

(In reply to comment #9)
> > It's a test outputs the result that Chromium perf bots can parse natively.
> 
> Is it a good format?  Should we change out PerformanceTests to match?

No, it's actually hideous looks like:
RESULT Bindings: event-target-wrapper= 1033.3 ms

RESULT panel-switch: show-panel-elements= 10 ms
RESULT panel-switch: show-panel-resources= 13 ms
RESULT panel-switch: show-panel-network= 38 ms
RESULT panel-switch: show-panel-scripts= 20 ms
RESULT panel-switch: show-panel-timeline= 12 ms
RESULT panel-switch: show-panel-profiles= 10 ms
RESULT panel-switch: show-panel-audits= 7 ms
RESULT panel-switch: show-panel-console= 13 ms
RESULT heap-delta: show-panel= 0 kB

[Ryosuke Niwa]

Committed r104856: <http://trac.webkit.org/changeset/104856>