Bug 44199

Summary: perf tests are flaky
Product: WebKit Reporter: Ojan Vafai <ojan>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, eric, mrobinson, rniwa, tony
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: PC   
OS: OS X 10.5   
Attachments:
Description Flags
Patch tony: review+

Ojan Vafai
Reported 2010-08-18 13:21:11 PDT
Some of the new perf tests seem to be flaky. I'll skip them on non-chromium platforms and mark them as PASS TEXT for chromium. That way, we can get flakiness data and make them non-flaky without turning bots red in in interim.
Attachments
Patch (2.70 KB, patch)
2010-08-24 15:04 PDT, Ojan Vafai 		tony: review+
DetailsFormatted DiffDiff
View All Add attachment proposed patch, testcase, etc.
Ojan Vafai
Comment 1 2010-08-24 15:04:53 PDT
Created attachment 65331 [details] Patch
Tony Chang
Comment 2 2010-08-24 15:18:18 PDT
Comment on attachment 65331 [details] Patch Ok for testing flakiness. If it works, we'll need to do this properly with layoutTestController, otherwise we can just revert this patch.
Ojan Vafai
Comment 3 2010-08-24 15:22:25 PDT
Committed r65940: <http://trac.webkit.org/changeset/65940>
Martin Robinson
Comment 4 2011-02-08 08:06:09 PST
Any updates on this?
Ojan Vafai
Comment 5 2011-02-09 22:30:31 PST
It did not fix the problem. I don't remember if it improved things though. Some of the tests are not flaky at all, but some of them are super flaky. See http://test-results.appspot.com/dashboards/flakiness_dashboard.html#master=ChromiumWebkit&tests=%5Eperf for the results of the flaky tests on the chromium bots. The ones that are not flaky don't show up there. Also, some of the ones listed there look like they crashed when all the tests in that run crashed (i.e. are not flaky). It's been vaguely on my TODO list to try and understand the cause of the flakiness, but I haven't gotten around to it.
Ojan Vafai
Comment 6 2011-02-09 22:31:17 PST
I suppose we could take the non-flaky ones off the Skipped lists.
Note You need to log in before you can comment on or make changes to this bug.