|Summary:||[GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily|
|Product:||WebKit||Reporter:||Michael Catanzaro <mcatanzaro>|
|Severity:||Normal||CC:||bugs-noreply, cgarcia, darin, mcatanzaro|
Description Michael Catanzaro 2016-04-09 09:31:16 PDT
Layout test fast/harness/sample-fail-mismatch-reftest.html is supposed to always fail, but it has been passing flakily for a while. It's started happening more frequently since r193683. This probably indicates a bug in our test harness.
Comment 2 Carlos Garcia Campos 2016-05-24 05:57:57 PDT
This is pretty weird because sample-fail-mismatch-reftest.html and sample-fail-mismatch-reftest-expected-mismatch.html are exactly the same file, so they should produce the same output. It just contains a line of text, no html, nor js, so I don't see how r193683 can affect this in any way. But in any case, if it's flaky for whatever reason the right expectations should be [ ImageOnlyFailure Pass ] not [ Failure Pass ]. In this case Pass means it failed and ImageOnlyFailure that it passed. Updated expectations in r201329