Bug 36619 - new-run-webkit-tests should use a test_expectations file in addition to Skipped files so we can handle pixel failures
Summary: new-run-webkit-tests should use a test_expectations file in addition to Skipp...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Dirk Pranke
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-03-25 14:31 PDT by Dirk Pranke
Modified: 2010-03-26 17:15 PDT (History)
4 users (show)

See Also:


Attachments
patch (6.89 KB, patch)
2010-03-25 14:41 PDT, Dirk Pranke
eric: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Pranke 2010-03-25 14:31:43 PDT
Skipped files give us no way to track pixel failures separately from other types of failures. If we start using a test_expectations.txt file we can track these and get a clean test run both with and without pixel tests enabled.
Comment 1 Dirk Pranke 2010-03-25 14:41:05 PDT
Created attachment 51677 [details]
patch
Comment 2 Eric Seidel (no email) 2010-03-25 14:42:29 PDT
Comment on attachment 51677 [details]
patch

Maybe that should go in mac-leopard?  LGTM.
Comment 3 Dirk Pranke 2010-03-25 14:58:01 PDT
test_expectations files should generally be as portable as possible, because maintaining multiple versions is a hassle, and one hopes that most bugs are cross-platform/cross-version, rather than version-specific.
Comment 4 Dirk Pranke 2010-03-26 17:15:30 PDT
Committed r56648: <http://trac.webkit.org/changeset/56648>