Bug 44199

Summary: perf tests are flaky
Product: WebKit Reporter: Ojan Vafai <ojan>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, eric, mrobinson, rniwa, tony
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: PC   
OS: OS X 10.5   
Attachments:
Description Flags
Patch tony: review+

Description Ojan Vafai 2010-08-18 13:21:11 PDT 
Some of the new perf tests seem to be flaky. I'll skip them on non-chromium platforms and mark them as PASS TEXT for chromium. That way, we can get flakiness data and make them non-flaky without turning bots red in in interim.
Comment 1 Ojan Vafai 2010-08-24 15:04:53 PDT 
Created attachment 65331 [details]
Patch
Comment 2 Tony Chang 2010-08-24 15:18:18 PDT 
Comment on attachment 65331 [details]
Patch

Ok for testing flakiness.  If it works, we'll need to do this properly with layoutTestController, otherwise we can just revert this patch.
Comment 3 Ojan Vafai 2010-08-24 15:22:25 PDT 
Committed r65940: <http://trac.webkit.org/changeset/65940>
Comment 4 Martin Robinson 2011-02-08 08:06:09 PST 
Any updates on this?
Comment 5 Ojan Vafai 2011-02-09 22:30:31 PST 
It did not fix the problem. I don't remember if it improved things though. Some of the tests are not flaky at all, but some of them are super flaky. See http://test-results.appspot.com/dashboards/flakiness_dashboard.html#master=ChromiumWebkit&tests=%5Eperf for the results of the flaky tests on the chromium bots. The ones that are not flaky don't show up there. Also, some of the ones listed there look like they crashed when all the tests in that run crashed (i.e. are not flaky).

It's been vaguely on my TODO list to try and understand the cause of the flakiness, but I haven't gotten around to it.
Comment 6 Ojan Vafai 2011-02-09 22:31:17 PST 
I suppose we could take the non-flaky ones off the Skipped lists.