Bug 79593 – [Performance Tests] [Chromium] Bindings/dom-attributes.html is timing out

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Ryosuke Niwa]

http://build.webkit.org/builders/Chromium%20Mac%20Release%20%28Perf%29/builds/530/steps/perf-test/logs/stdio

Running Bindings/dom-attributes.html (1 of 44)
timeout: PerformanceTests/Bindings/dom-attributes.html
FAILED

Letting any test timeout is incredibly harmful on perf bots because we let each test run 10 minutes.

[Ryosuke Niwa]

Given the test is generating passing on Lion, I bet this is a bug in Chromium code or Chromium DumpRenderTree:

http://build.webkit.org/builders/Lion%20Intel%20Release%20%28Perf%29/builds/459/steps/perf-test/logs/stdio
RESULT Bindings: dom-attributes= 8829.45 ms
median= 8839.5 ms, stdev= 32.2327085427 ms, min= 8770.0 ms, max= 8871.0 ms

[Kentaro Hara]

Created attachment 128934 [details]
Patch

[Ryosuke Niwa]

Comment on attachment 128934 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=128934&action=review

> PerformanceTests/ChangeLog:10
> +        V8 DOM bindings are too much slower than JSC bindings.

That's really bad. We should fix that. The results are also not reliable on Mac/Qt:
http://webkit-perf.appspot.com/graph.html#tests=[[150153,2001,100137],[150153,2001,32196]]&sel=1330111816007.1082,1330303636915&displayrange=7&datatype=running

Could you investigate why that's the case? Do we just need more samples or are we doing something inherently unstable?

[Kentaro Hara]

(In reply to comment #3)
> > PerformanceTests/ChangeLog:10
> > +        V8 DOM bindings are too much slower than JSC bindings.
> 
> That's really bad. We should fix that.

Yes, yes, yes, really. I've been investigating why (using my 20% time though).

> The results are also not reliable on Mac/Qt:
> http://webkit-perf.appspot.com/graph.html#tests=[[150153,2001,100137],[150153,2001,32196]]&sel=1330111816007.1082,1330303636915&displayrange=7&datatype=running
> 
> Could you investigate why that's the case? Do we just need more samples or are we doing something inherently unstable?

OK. But let me commit the patch to observe the reliability on the perf bots.

[Ryosuke Niwa]

(In reply to comment #4)
> (In reply to comment #3)
> > > PerformanceTests/ChangeLog:10
> > > +        V8 DOM bindings are too much slower than JSC bindings.
> > 
> > That's really bad. We should fix that.
> 
> Yes, yes, yes, really. I've been investigating why (using my 20% time though).
> 
> > The results are also not reliable on Mac/Qt:
> > http://webkit-perf.appspot.com/graph.html#tests=[[150153,2001,100137],[150153,2001,32196]]&sel=1330111816007.1082,1330303636915&displayrange=7&datatype=running
> > 
> > Could you investigate why that's the case? Do we just need more samples or are we doing something inherently unstable?
> 
> OK. But let me commit the patch to observe the reliability on the perf bots.

Sounds good to me.

[WebKit Review Bot]

Comment on attachment 128934 [details]
Patch

Attachment 128934 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/11628770

New failing tests:
inspector/protocol/console-agent.html

[WebKit Review Bot]

Comment on attachment 128934 [details]
Patch

Clearing flags on attachment: 128934

Committed r108942: <http://trac.webkit.org/changeset/108942>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.