Bug 102500 - DOM/DOMWalk.html is flaky in Chromium Win Release Perf bot
Summary: DOM/DOMWalk.html is flaky in Chromium Win Release Perf bot
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Dominic Cooney
URL:
Keywords:
: 104866 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-11-16 06:40 PST by Kentaro Hara
Modified: 2013-01-15 01:43 PST (History)
4 users (show)

See Also:


Attachments
Patch (1.49 KB, patch)
2013-01-15 00:59 PST, Dominic Cooney
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kentaro Hara 2012-11-16 06:40:46 PST
It sometimes show the following error:

Running DOM/DOMWalk.html (35 of 112)
runFunction returned a non-positive value: 0
runFunction returned a non-positive value: 0
FAILED
Finished: 9.609000 s

See http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Perf%29/builds/4412/steps/perf-test/logs/stdio
Comment 1 Dominic Cooney 2013-01-15 00:30:51 PST
*** Bug 104866 has been marked as a duplicate of this bug. ***
Comment 2 Dominic Cooney 2013-01-15 00:59:15 PST
Created attachment 182715 [details]
Patch
Comment 3 Dominic Cooney 2013-01-15 01:03:11 PST
I am assuming that zero is a plausible result for some of the (micro) benchmarks in DOM/DOMWalk.html.

One thing that makes me suspicious is that Dromaeo results on the Windows bots are typically numerically larger than those of the Mac bots, yet the Mac bot does not exhibit this failure. Perhaps the timer works differently on those platforms.

If this warrants more investigation, the alternative is to add more instrumentation to Dromaeo so we can see which specific test functions are failing (it is a flaky set, usually >1 but from zero to a few tests fail this way.)

For now proposing that we assume the result is correct and allow zero as a (very fast) result to green the Windows perf bot.
Comment 4 Kentaro Hara 2013-01-15 01:11:31 PST
Comment on attachment 182715 [details]
Patch

Looks reasonable to me as an immediate fix to suppress the win bot errors, though we want to improve the test so it won't return zero (zero means that we're not testing something meaningful).
Comment 5 Dominic Cooney 2013-01-15 01:17:28 PST
Comment on attachment 182715 [details]
Patch

Thank you for the quick review.

I filed bug 106882 to improve the test.
Comment 6 WebKit Review Bot 2013-01-15 01:43:51 PST
Comment on attachment 182715 [details]
Patch

Clearing flags on attachment: 182715

Committed r139728: <http://trac.webkit.org/changeset/139728>
Comment 7 WebKit Review Bot 2013-01-15 01:43:54 PST
All reviewed patches have been landed.  Closing bug.