SheriffBot should announce when the commit-queue hits a flaky test Flaky tests should be as annoying for the channel as they are for folks trying to use the commit-queue. We also really just need the flakey test dashboard.
How do you distinguish the commit queue hitting a flaky test versus a test that actually fails due to the patch it's trying to commit? As far as the flakiness dashboard, getting it hooked up is just a matter of finally switching over to new-run-webkit-tests. Alternately, making the JSON generation code work standalone (shouldn't be too hard) and having old-run-webkit-tests call out to it.
(In reply to comment #1) > How do you distinguish the commit queue hitting a flaky test versus a test that actually fails due to the patch it's trying to commit? > > As far as the flakiness dashboard, getting it hooked up is just a matter of finally switching over to new-run-webkit-tests. Alternately, making the JSON generation code work standalone (shouldn't be too hard) and having old-run-webkit-tests call out to it. +1 to switching to NWRT.
I think this is better covered by bug 50803. *** This bug has been marked as a duplicate of bug 50803 ***