Bug 45740
| Summary: | SheriffBot should announce when the commit-queue hits a flaky test | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Eric Seidel (no email) <eric> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | Normal | CC: | abarth, dglazkov, dpranke, ojan, victorw |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | PC | ||
| OS: | OS X 10.5 | ||
Eric Seidel (no email)
SheriffBot should announce when the commit-queue hits a flaky test
Flaky tests should be as annoying for the channel as they are for folks trying to use the commit-queue.
We also really just need the flakey test dashboard.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Ojan Vafai
How do you distinguish the commit queue hitting a flaky test versus a test that actually fails due to the patch it's trying to commit?
As far as the flakiness dashboard, getting it hooked up is just a matter of finally switching over to new-run-webkit-tests. Alternately, making the JSON generation code work standalone (shouldn't be too hard) and having old-run-webkit-tests call out to it.
Dimitri Glazkov (Google)
(In reply to comment #1)
> How do you distinguish the commit queue hitting a flaky test versus a test that actually fails due to the patch it's trying to commit?
>
> As far as the flakiness dashboard, getting it hooked up is just a matter of finally switching over to new-run-webkit-tests. Alternately, making the JSON generation code work standalone (shouldn't be too hard) and having old-run-webkit-tests call out to it.
+1 to switching to NWRT.
Eric Seidel (no email)
I think this is better covered by bug 50803.
*** This bug has been marked as a duplicate of bug 50803 ***