Bug 45740 - SheriffBot should announce when the commit-queue hits a flaky test
Summary: SheriffBot should announce when the commit-queue hits a flaky test
Status: RESOLVED DUPLICATE of bug 50803
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-09-14 04:01 PDT by Eric Seidel (no email)
Modified: 2010-12-10 17:36 PST (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2010-09-14 04:01:39 PDT
SheriffBot should announce when the commit-queue hits a flaky test

Flaky tests should be as annoying for the channel as they are for folks trying to use the commit-queue.

We also really just need the flakey test dashboard.
Comment 1 Ojan Vafai 2010-09-14 05:11:54 PDT
How do you distinguish the commit queue hitting a flaky test versus a test that actually fails due to the patch it's trying to commit?

As far as the flakiness dashboard, getting it hooked up is just a matter of finally switching over to new-run-webkit-tests. Alternately, making the JSON generation code work standalone (shouldn't be too hard) and having old-run-webkit-tests call out to it.
Comment 2 Dimitri Glazkov (Google) 2010-09-14 07:35:14 PDT
(In reply to comment #1)
> How do you distinguish the commit queue hitting a flaky test versus a test that actually fails due to the patch it's trying to commit?
> 
> As far as the flakiness dashboard, getting it hooked up is just a matter of finally switching over to new-run-webkit-tests. Alternately, making the JSON generation code work standalone (shouldn't be too hard) and having old-run-webkit-tests call out to it.

+1 to switching to NWRT.
Comment 3 Eric Seidel (no email) 2010-12-10 17:36:31 PST
I think this is better covered by bug 50803.

*** This bug has been marked as a duplicate of bug 50803 ***