Bug 156436 - [GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily
Summary: [GTK] fast/harness/sample-fail-mismatch-reftest.html passes flakily
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKitGTK (show other bugs)
Version: Other
Hardware: PC Linux
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-04-09 09:31 PDT by Michael Catanzaro
Modified: 2016-05-24 05:57 PDT (History)
4 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Catanzaro 2016-04-09 09:31:16 PDT 
Layout test fast/harness/sample-fail-mismatch-reftest.html is supposed to always fail, but it has been passing flakily for a while. It's started happening more frequently since r193683. This probably indicates a bug in our test harness.
Comment 1 Michael Catanzaro 2016-04-09 10:31:58 PDT 
Updated expectations in r199270.
Comment 2 Carlos Garcia Campos 2016-05-24 05:57:57 PDT 
This is pretty weird because sample-fail-mismatch-reftest.html and sample-fail-mismatch-reftest-expected-mismatch.html are exactly the same file, so they should produce the same output. It just contains a line of text, no html, nor js, so I don't see how r193683 can affect this in any way. But in any case, if it's flaky for whatever reason the right expectations should be [ ImageOnlyFailure Pass ] not [ Failure Pass ]. In this case Pass means it failed and ImageOnlyFailure that it passed. Updated expectations in r201329