Bug 66989 - Need to triage transforms/3d layout test results
Summary: Need to triage transforms/3d layout test results
Status: RESOLVED WONTFIX
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Shawn Singh
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-25 15:31 PDT by Peter Kasting
Modified: 2013-04-10 13:09 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Kasting 2011-08-25 15:31:37 PDT
Chromium's test expectations were skipping the "transforms/3d" directory with the claim that it was a "work in progress" that no one (including Apple) was running, and that we should re-enable once anyone was running it.

According to the current waterfall, Apple is running these tests now for the Mac bots, so I'm unskipping the directory (but marking it PASS FAIL).  We should triage the test results in here to see what actions need to be taken.
Comment 1 Levi Weintraub 2012-06-01 15:18:46 PDT
This pass/fail test expectation masked a bug in our sub-pixel work. Any chance we can get these looked at? Nudge nudge ;)
Comment 2 Shawn Singh 2012-06-01 15:28:39 PDT
dude...
Comment 3 Shawn Singh 2012-06-01 16:08:33 PDT
Update:  there are no serious fires.  most tests fell into 2 categories:
  (1) need rebaselining
  (2) a known layer sorting issue with intersecting layers


only 3-4 tests were questionable otherwise, and hopefully can be addressed without panic.

I'll officially triage things early next week.
Comment 4 Shawn Singh 2012-06-01 16:10:07 PDT
I do have one question, in the mean-time.

Is there any way we can have a state called "this is an incorrect baseline, but we want to keep running the test for now, so that the test is still useful" ?

That would be nice to have for the results of type (2), because that known issue may not be solved any time soon, but the tests are still worth running otherwise.
Comment 5 Peter Kasting 2012-06-01 16:12:21 PDT
(In reply to comment #4)
> Is there any way we can have a state called "this is an incorrect baseline, but we want to keep running the test for now, so that the test is still useful" ?

I suspect you could check in an expectation like "BUG123 : foo.html = PASS", with a comment above it explaining why it's around?

Or just have a bug on file and check in the "bad" baseline without a test expectation.

In either case, the onus is really on the bug assignee to make suire this eventually gets solved and not lost.
Comment 6 Shawn Singh 2012-06-03 22:18:35 PDT
First round of rebaselines is committed: http://trac.webkit.org/changeset/119369

The rest of them I will continue to look at.  For now I marked them as FAIL instead of PASS FAIL.

There were 3 bugs I think I'm seeing in the incorrect results:

  (1) awkward clipping bug.  This might be fixed by existing known issue https://bugs.webkit.org/show_bug.cgi?id=85808, otherwise I'll create a bug on it.

  (2) something weird going on with perspective on a few of the cases, where the elements seem to get zoomed in further in chrome than on the expected result

  (3) the known layer sorting result.  As discussed, I'll go ahead and rebaseline those anyway since this bug will unfortunately be unsolved for a while.
Comment 7 Shawn Singh 2013-04-10 13:09:52 PDT
This issue no longer applies, it's only for chromium/blink side.