Bug 102500

Summary: DOM/DOMWalk.html is flaky in Chromium Win Release Perf bot
Product: WebKit Reporter: Kentaro Hara <haraken>
Component: Tools / TestsAssignee: Dominic Cooney <dominicc>
Severity: Normal CC: dominicc, noel.gordon, rniwa, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Description Flags
Patch none

Description Kentaro Hara 2012-11-16 06:40:46 PST
It sometimes show the following error:

Running DOM/DOMWalk.html (35 of 112)
runFunction returned a non-positive value: 0
runFunction returned a non-positive value: 0
Finished: 9.609000 s

See http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Perf%29/builds/4412/steps/perf-test/logs/stdio
Comment 1 Dominic Cooney 2013-01-15 00:30:51 PST
*** Bug 104866 has been marked as a duplicate of this bug. ***
Comment 2 Dominic Cooney 2013-01-15 00:59:15 PST
Created attachment 182715 [details]
Comment 3 Dominic Cooney 2013-01-15 01:03:11 PST
I am assuming that zero is a plausible result for some of the (micro) benchmarks in DOM/DOMWalk.html.

One thing that makes me suspicious is that Dromaeo results on the Windows bots are typically numerically larger than those of the Mac bots, yet the Mac bot does not exhibit this failure. Perhaps the timer works differently on those platforms.

If this warrants more investigation, the alternative is to add more instrumentation to Dromaeo so we can see which specific test functions are failing (it is a flaky set, usually >1 but from zero to a few tests fail this way.)

For now proposing that we assume the result is correct and allow zero as a (very fast) result to green the Windows perf bot.
Comment 4 Kentaro Hara 2013-01-15 01:11:31 PST
Comment on attachment 182715 [details]

Looks reasonable to me as an immediate fix to suppress the win bot errors, though we want to improve the test so it won't return zero (zero means that we're not testing something meaningful).
Comment 5 Dominic Cooney 2013-01-15 01:17:28 PST
Comment on attachment 182715 [details]

Thank you for the quick review.

I filed bug 106882 to improve the test.
Comment 6 WebKit Review Bot 2013-01-15 01:43:51 PST
Comment on attachment 182715 [details]

Clearing flags on attachment: 182715

Committed r139728: <http://trac.webkit.org/changeset/139728>
Comment 7 WebKit Review Bot 2013-01-15 01:43:54 PST
All reviewed patches have been landed.  Closing bug.