Bug 113223 – REGRESSION(r146657): Test failures reported incorrectly

• Component: New Bugs
• Status: RESOLVED FIXED

[Csaba Osztrogonác]

After http://trac.webkit.org/changeset/146657 simple test 
failures reported incorrectly by NRWT and buildbot.

Here is a simple example: css3/filters/effect-reference.html 
was broken by http://trac.webkit.org/changeset/146779.
(It seems it needs only rebaseline.)

But the problem is the following NRWT output:
...
09:52:43.778 10859 [4003/28958] css3/filters/effect-reference.html failed unexpectedly (text diff)
09:52:43.778 10859 worker/0 css3/filters/effect-reference.html failed:
09:52:43.778 10859 worker/0  text diff
...
10:11:28.026 10859 Retrying 6 unexpected failure(s) ...
...
10:20:17.156 10859 [4/6] css3/filters/effect-reference.html failed unexpectedly (text diff, image diff)
10:20:17.156 10859 worker/0 css3/filters/effect-reference.html failed:
10:20:17.157 10859 worker/0  text diff
10:20:17.157 10859 worker/0  image diff
...
10:20:27.659 10859 Testing completed, Exit status: 1 <-------- correct exit code
...
Unexpected flakiness: text-only failures (4)
  css3/filters/effect-reference.html [ Failure Pass Failure ]   <------ This line is absolutely wrong!
...


and the reported result by the bot: (It is wrong, because its input is wrong)
9 new passes 6 flakes 7 missing results 

See http://build.webkit.org/builders/Qt%20Linux%20Release/builds/58743 for details

[Ryosuke Niwa]

Created attachment 194912 [details]
Fixes the bug

[Build Bot]

Comment on attachment 194912 [details]
Fixes the bug

Attachment 194912 [details] did not pass win-ews (win):
Output: http://webkit-commit-queue.appspot.com/results/17255406

[Ryosuke Niwa]

Committed r146817: <http://trac.webkit.org/changeset/146817>