Bug 156436

Summary: [GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily
Product: WebKit Reporter: Michael Catanzaro <mcatanzaro>
Component: WebKitGTKAssignee: Nobody <webkit-unassigned>
Status: NEW    
Severity: Normal CC: bugs-noreply, cgarcia, darin, mcatanzaro
Priority: P2    
Version: Other   
Hardware: PC   
OS: Linux   

Michael Catanzaro
Reported 2016-04-09 09:31:16 PDT
Layout test fast/harness/sample-fail-mismatch-reftest.html is supposed to always fail, but it has been passing flakily for a while. It's started happening more frequently since r193683. This probably indicates a bug in our test harness.
Attachments
Michael Catanzaro
Comment 1 2016-04-09 10:31:58 PDT
Updated expectations in r199270.
Carlos Garcia Campos
Comment 2 2016-05-24 05:57:57 PDT
This is pretty weird because sample-fail-mismatch-reftest.html and sample-fail-mismatch-reftest-expected-mismatch.html are exactly the same file, so they should produce the same output. It just contains a line of text, no html, nor js, so I don't see how r193683 can affect this in any way. But in any case, if it's flaky for whatever reason the right expectations should be [ ImageOnlyFailure Pass ] not [ Failure Pass ]. In this case Pass means it failed and ImageOnlyFailure that it passed. Updated expectations in r201329
Note You need to log in before you can comment on or make changes to this bug.