Bug 66989 – Need to triage transforms/3d layout test results

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Peter Kasting]

Chromium's test expectations were skipping the "transforms/3d" directory with the claim that it was a "work in progress" that no one (including Apple) was running, and that we should re-enable once anyone was running it.

According to the current waterfall, Apple is running these tests now for the Mac bots, so I'm unskipping the directory (but marking it PASS FAIL).  We should triage the test results in here to see what actions need to be taken.

[Levi Weintraub]

This pass/fail test expectation masked a bug in our sub-pixel work. Any chance we can get these looked at? Nudge nudge ;)

[Shawn Singh]

dude...

[Shawn Singh]

Update:  there are no serious fires.  most tests fell into 2 categories:
  (1) need rebaselining
  (2) a known layer sorting issue with intersecting layers


only 3-4 tests were questionable otherwise, and hopefully can be addressed without panic.

I'll officially triage things early next week.

[Shawn Singh]

I do have one question, in the mean-time.

Is there any way we can have a state called "this is an incorrect baseline, but we want to keep running the test for now, so that the test is still useful" ?

That would be nice to have for the results of type (2), because that known issue may not be solved any time soon, but the tests are still worth running otherwise.

[Peter Kasting]

(In reply to comment #4)
> Is there any way we can have a state called "this is an incorrect baseline, but we want to keep running the test for now, so that the test is still useful" ?

I suspect you could check in an expectation like "BUG123 : foo.html = PASS", with a comment above it explaining why it's around?

Or just have a bug on file and check in the "bad" baseline without a test expectation.

In either case, the onus is really on the bug assignee to make suire this eventually gets solved and not lost.

[Shawn Singh]

First round of rebaselines is committed: http://trac.webkit.org/changeset/119369

The rest of them I will continue to look at.  For now I marked them as FAIL instead of PASS FAIL.

There were 3 bugs I think I'm seeing in the incorrect results:

  (1) awkward clipping bug.  This might be fixed by existing known issue https://bugs.webkit.org/show_bug.cgi?id=85808, otherwise I'll create a bug on it.

  (2) something weird going on with perspective on a few of the cases, where the elements seem to get zoomed in further in chrome than on the expected result

  (3) the known layer sorting result.  As discussed, I'll go ahead and rebaseline those anyway since this bug will unfortunately be unsolved for a while.

[Shawn Singh]

This issue no longer applies, it's only for chromium/blink side.