Bug 69279 – allow failure of most Chromium-Mac-Skia tests while we clean up that port

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

caryclark and epoger are hard at work getting the Chromium-Mac-Skia port stable. This will make the test results unstable for a while.  Disabling most tests of this port so as not to distract Webkit Gardeners with excess noise (as happened in https://bugs.webkit.org/show_bug.cgi?id=69166 )

Created attachment 109524 [details]
Patch

I have confirmed locally that this would have hidden all layout test failures caused by the CL in https://bugs.webkit.org/show_bug.cgi?id=69166 .

After it passes the trybots, I will request review from a human...

[Adam Barth]

Comment on attachment 109524 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=109524&action=review

What about LEOPARD?  Probably all these SNOWLEOPARDs should just be MAC.

> LayoutTests/platform/chromium/test_expectations.txt:3699
> +BUGWK69279 SNOWLEOPARD CPU : media = PASS TEXT IMAGE IMAGE+TEXT CRASH

No GPU here?

> LayoutTests/platform/chromium/test_expectations.txt:3706
> +BUGWK69279 SNOWLEOPARD CPU GPU : transitions = PASS TEXT IMAGE IMAGE+TEXT CRASH

I think "TEXT IMAGE IMAGE+TEXT" and all be combined into FAIL.

[Adam Barth]

> I think "TEXT IMAGE IMAGE+TEXT" and all be combined into FAIL.

Actually, CRASH is covered too.  You can just mark these as PASS FAIL.  (I'm happy to take another look at this patch if you'd like me to before you land it.)

Created attachment 109531 [details]
Patch

Comment on attachment 109524 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=109524&action=review

>> LayoutTests/platform/chromium/test_expectations.txt:3699
>> +BUGWK69279 SNOWLEOPARD CPU : media = PASS TEXT IMAGE IMAGE+TEXT CRASH
> 
> No GPU here?

Added.  (Previously, I was mindlessly copying from the Leopard section above)

>> LayoutTests/platform/chromium/test_expectations.txt:3706
>> +BUGWK69279 SNOWLEOPARD CPU GPU : transitions = PASS TEXT IMAGE IMAGE+TEXT CRASH
> 
> I think "TEXT IMAGE IMAGE+TEXT" and all be combined into FAIL.

Done.

(In reply to comment #3)
> (From update of attachment 109524 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=109524&action=review
> 
> What about LEOPARD?  Probably all these SNOWLEOPARDs should just be MAC.

I'd like to keep the LEOPARD and SNOWLEOPARD sections separate if that's OK with you.  (I think we can turn them back on at different times.)

[Adam Barth]

Comment on attachment 109531 [details]
Patch

Looks great.

[Adam Barth]

Comment on attachment 109531 [details]
Patch

I don't understand.  If you make a change that causes these tests to fail on both LEOPARD and SNOWLEOPARD, the SNOWLEOPARD bot will stay green but the LEOPARD bot will go red.  Is that what you intend?

(In reply to comment #9)
> (From update of attachment 109531 [details])
> I don't understand.  If you make a change that causes these tests to fail on both LEOPARD and SNOWLEOPARD, the SNOWLEOPARD bot will stay green but the LEOPARD bot will go red.  Is that what you intend?

There is already a section of the test_expectations file that deals with the LEOPARD tests, allowing most of them to fail for now.

[WebKit Review Bot]

Comment on attachment 109531 [details]
Patch

Clearing flags on attachment: 109531

Committed r96592: <http://trac.webkit.org/changeset/96592>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.