Bug 156436

Summary: [GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily
Product: WebKit Reporter: Michael Catanzaro <mcatanzaro>
Component: WebKitGTKAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: bugs-noreply, cgarcia, darin, mcatanzaro
Priority: P2    
Version: Other   
Hardware: PC   
OS: Linux   

Description Michael Catanzaro 2016-04-09 09:31:16 PDT 
Layout test fast/harness/sample-fail-mismatch-reftest.html is supposed to always fail, but it has been passing flakily for a while. It's started happening more frequently since r193683. This probably indicates a bug in our test harness.
Comment 1 Michael Catanzaro 2016-04-09 10:31:58 PDT 
Updated expectations in r199270.
Comment 2 Carlos Garcia Campos 2016-05-24 05:57:57 PDT 
This is pretty weird because sample-fail-mismatch-reftest.html and sample-fail-mismatch-reftest-expected-mismatch.html are exactly the same file, so they should produce the same output. It just contains a line of text, no html, nor js, so I don't see how r193683 can affect this in any way. But in any case, if it's flaky for whatever reason the right expectations should be [ ImageOnlyFailure Pass ] not [ Failure Pass ]. In this case Pass means it failed and ImageOnlyFailure that it passed. Updated expectations in r201329