NEW 119749
Tests often incorrectly reported as passing 
https://bugs.webkit.org/show_bug.cgi?id=119749
Summary Tests often incorrectly reported as passing
Eric Carlson
Reported 2013-08-13 08:09:23 PDT
The build bots often incorrectly report tests as "expected to fail but passed". For example http/tests/navigation/back-twice-without-commit.html was marked as Timeout in Mac TestExpectations, and the MountainLion bot showed that it was passing unexpectedly: http://build.webkit.org/results/Apple%20MountainLion%20Debug%20WK2%20(Tests)/r153825%20(11521)/results.html but the flakiness dashboard shows that it never passes: http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fnavigation%2Fback-twice-without-commit.html
Attachments
Add attachment proposed patch, testcase, etc.
Alexey Proskuryakov
Comment 1 2013-08-13 09:46:15 PDT
I’m not sure if there is anything wrong with the tools - the bot where the test passes is a WK2 one, and the flakiness dashboard only complains about WK1 bots. Dashboard output is quite confusing though - as recently as yesterday afternoon, I misunderstood which bots it followed.
Note You need to log in before you can comment on or make changes to this bug.