NEW 156436
[GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily 
https://bugs.webkit.org/show_bug.cgi?id=156436
Summary [GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily
Michael Catanzaro
Reported 2016-04-09 09:31:16 PDT
Layout test fast/harness/sample-fail-mismatch-reftest.html is supposed to always fail, but it has been passing flakily for a while. It's started happening more frequently since r193683. This probably indicates a bug in our test harness.
Attachments
Add attachment proposed patch, testcase, etc.
Michael Catanzaro
Comment 1 2016-04-09 10:31:58 PDT
Updated expectations in r199270.
Carlos Garcia Campos
Comment 2 2016-05-24 05:57:57 PDT
This is pretty weird because sample-fail-mismatch-reftest.html and sample-fail-mismatch-reftest-expected-mismatch.html are exactly the same file, so they should produce the same output. It just contains a line of text, no html, nor js, so I don't see how r193683 can affect this in any way. But in any case, if it's flaky for whatever reason the right expectations should be [ ImageOnlyFailure Pass ] not [ Failure Pass ]. In this case Pass means it failed and ImageOnlyFailure that it passed. Updated expectations in r201329
Note You need to log in before you can comment on or make changes to this bug.