Bug 170898

Summary: [selectors4] Import W3C Test Suite
Product: WebKit Reporter: Manuel Rego Casasnovas <rego>
Component: Tools / TestsAssignee: Manuel Rego Casasnovas <rego>
Status: RESOLVED FIXED    
Severity: Normal CC: benjamin, buildbot, commit-queue, lforschler, rniwa, ryanhaddad, youennf
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 170899, 170900, 170935    
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews100 for mac-elcapitan
none
Archive of layout-test-results from ews113 for mac-elcapitan
none
Patch for landing none

Description Manuel Rego Casasnovas 2017-04-17 01:26:33 PDT
It'd be nice to import this test suite as it spots an issue on current :focus-within implementation.

The suite is at WPT repo:
https://github.com/w3c/web-platform-tests/tree/master/css/selectors4
Comment 1 Manuel Rego Casasnovas 2017-04-17 01:43:33 PDT
Created attachment 307262 [details]
Patch
Comment 2 Build Bot 2017-04-17 02:57:39 PDT
Comment on attachment 307262 [details]
Patch

Attachment 307262 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/3550192

New failing tests:
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-003.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-001.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-004.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-005.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-002.html
Comment 3 Build Bot 2017-04-17 02:57:40 PDT
Created attachment 307264 [details]
Archive of layout-test-results from ews100 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 4 Build Bot 2017-04-17 03:06:21 PDT
Comment on attachment 307262 [details]
Patch

Attachment 307262 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/3550199

New failing tests:
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-003.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-001.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-004.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-005.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-002.html
Comment 5 Build Bot 2017-04-17 03:06:22 PDT
Created attachment 307265 [details]
Archive of layout-test-results from ews113 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 6 Manuel Rego Casasnovas 2017-04-17 04:28:05 PDT
It seems the Mac EWSs are failing only in the Shadow DOM tests:

(In reply to Build Bot from comment #2)
> New failing tests:
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-003.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-001.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-004.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-005.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-002.html

I've tried to reproduce the failure on a macpro, but I couldn't manage to do it.
In my case these tests are passing properly, also "mac-wk2" EWS is green.

Could someone give me any hint on what's going on here? Thanks!
Comment 7 youenn fablet 2017-04-17 07:18:40 PDT
Not sure why they fail.
You can add specific mac-wk1 [Pass Failure] TestExpectations for these I guess.
Comment 8 Manuel Rego Casasnovas 2017-04-18 00:55:49 PDT
Created attachment 307357 [details]
Patch for landing
Comment 9 Manuel Rego Casasnovas 2017-04-18 01:01:27 PDT
(In reply to youenn fablet from comment #7)
> Not sure why they fail.
> You can add specific mac-wk1 [Pass Failure] TestExpectations for these I
> guess.

It seems a timing issue, I've added more info in bug #170935.
I can reproduce the issue now passing "-1" to run-webkit-tests in order to use WK1.
Anyway I'll skip them for the first import and then work on getting them unskipped.

Thanks for the review.
Comment 10 WebKit Commit Bot 2017-04-18 02:17:23 PDT
Comment on attachment 307357 [details]
Patch for landing

Clearing flags on attachment: 307357

Committed r215457: <http://trac.webkit.org/changeset/215457>
Comment 11 WebKit Commit Bot 2017-04-18 02:17:25 PDT
All reviewed patches have been landed.  Closing bug.
Comment 12 Ryan Haddad 2017-04-18 09:33:56 PDT
It looks like the tests should have been marked as 'ImageOnlyFailure' for mac-wk1. I took care of that in https://trac.webkit.org/changeset/215464/webkit.
Comment 13 Manuel Rego Casasnovas 2017-04-18 23:47:12 PDT
(In reply to Ryan Haddad from comment #12)
> It looks like the tests should have been marked as 'ImageOnlyFailure' for
> mac-wk1. I took care of that in
> https://trac.webkit.org/changeset/215464/webkit.

Oops, sorry about that. Thanks for fixing it.