Bug 44199 - perf tests are flaky
Summary: perf tests are flaky
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Nobody
Depends on:
Reported: 2010-08-18 13:21 PDT by Ojan Vafai
Modified: 2011-02-09 22:31 PST (History)
5 users (show)

See Also:

Patch (2.70 KB, patch)
2010-08-24 15:04 PDT, Ojan Vafai
tony: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ojan Vafai 2010-08-18 13:21:11 PDT
Some of the new perf tests seem to be flaky. I'll skip them on non-chromium platforms and mark them as PASS TEXT for chromium. That way, we can get flakiness data and make them non-flaky without turning bots red in in interim.
Comment 1 Ojan Vafai 2010-08-24 15:04:53 PDT
Created attachment 65331 [details]
Comment 2 Tony Chang 2010-08-24 15:18:18 PDT
Comment on attachment 65331 [details]

Ok for testing flakiness.  If it works, we'll need to do this properly with layoutTestController, otherwise we can just revert this patch.
Comment 3 Ojan Vafai 2010-08-24 15:22:25 PDT
Committed r65940: <http://trac.webkit.org/changeset/65940>
Comment 4 Martin Robinson 2011-02-08 08:06:09 PST
Any updates on this?
Comment 5 Ojan Vafai 2011-02-09 22:30:31 PST
It did not fix the problem. I don't remember if it improved things though. Some of the tests are not flaky at all, but some of them are super flaky. See http://test-results.appspot.com/dashboards/flakiness_dashboard.html#master=ChromiumWebkit&tests=%5Eperf for the results of the flaky tests on the chromium bots. The ones that are not flaky don't show up there. Also, some of the ones listed there look like they crashed when all the tests in that run crashed (i.e. are not flaky).

It's been vaguely on my TODO list to try and understand the cause of the flakiness, but I haven't gotten around to it.
Comment 6 Ojan Vafai 2011-02-09 22:31:17 PST
I suppose we could take the non-flaky ones off the Skipped lists.