Bug 66071
| Summary: | Teach garden-o-matic about flaky tests | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Dimitri Glazkov (Google) <dglazkov> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | abarth, tony |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Bug Depends on: | |||
| Bug Blocks: | 69227 | ||
Dimitri Glazkov (Google)
We have all the information necessary to recognize flaky tests. We should use it to prompt the gardener to mark the test as flaky.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Dimitri Glazkov (Google)
The flakiness should be revealed in the "Failing Tests" view, so that you could have a choice to mark a test as either failing or flaky.
It might also be useful to show an item in the ui.notifications.TestsFailing "causes" list, indicating that a test is flaky -- and provide a "Mark as Flaky" button there, too.
An edge case where you have 400 tests failing, and 300 of them are also flaky, thus making "causes" list be extremely long seems unlikely.
Adam Barth
Punt to v2.
Dimitri Glazkov (Google)
avoid a trip to the flakiness dashboard
- is the test flaky?
- since when?