Bug 170898 - [selectors4] Import W3C Test Suite
Summary: [selectors4] Import W3C Test Suite
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Manuel Rego Casasnovas
URL:
Keywords:
Depends on:
Blocks: 170899 170900 170935
  Show dependency treegraph
 
Reported: 2017-04-17 01:26 PDT by Manuel Rego Casasnovas
Modified: 2017-04-18 23:47 PDT (History)
7 users (show)

See Also:


Attachments
Patch (69.45 KB, patch)
2017-04-17 01:43 PDT, Manuel Rego Casasnovas
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews100 for mac-elcapitan (887.08 KB, application/zip)
2017-04-17 02:57 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews113 for mac-elcapitan (1.65 MB, application/zip)
2017-04-17 03:06 PDT, Build Bot
no flags Details
Patch for landing (70.65 KB, patch)
2017-04-18 00:55 PDT, Manuel Rego Casasnovas
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Manuel Rego Casasnovas 2017-04-17 01:26:33 PDT
It'd be nice to import this test suite as it spots an issue on current :focus-within implementation.

The suite is at WPT repo:
https://github.com/w3c/web-platform-tests/tree/master/css/selectors4
Comment 1 Manuel Rego Casasnovas 2017-04-17 01:43:33 PDT
Created attachment 307262 [details]
Patch
Comment 2 Build Bot 2017-04-17 02:57:39 PDT
Comment on attachment 307262 [details]
Patch

Attachment 307262 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/3550192

New failing tests:
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-003.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-001.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-004.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-005.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-002.html
Comment 3 Build Bot 2017-04-17 02:57:40 PDT
Created attachment 307264 [details]
Archive of layout-test-results from ews100 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 4 Build Bot 2017-04-17 03:06:21 PDT
Comment on attachment 307262 [details]
Patch

Attachment 307262 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/3550199

New failing tests:
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-003.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-001.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-004.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-005.html
imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-002.html
Comment 5 Build Bot 2017-04-17 03:06:22 PDT
Created attachment 307265 [details]
Archive of layout-test-results from ews113 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 6 Manuel Rego Casasnovas 2017-04-17 04:28:05 PDT
It seems the Mac EWSs are failing only in the Shadow DOM tests:

(In reply to Build Bot from comment #2)
> New failing tests:
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-003.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-001.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-004.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-005.html
> imported/w3c/web-platform-tests/css/selectors4/focus-within-shadow-002.html

I've tried to reproduce the failure on a macpro, but I couldn't manage to do it.
In my case these tests are passing properly, also "mac-wk2" EWS is green.

Could someone give me any hint on what's going on here? Thanks!
Comment 7 youenn fablet 2017-04-17 07:18:40 PDT
Not sure why they fail.
You can add specific mac-wk1 [Pass Failure] TestExpectations for these I guess.
Comment 8 Manuel Rego Casasnovas 2017-04-18 00:55:49 PDT
Created attachment 307357 [details]
Patch for landing
Comment 9 Manuel Rego Casasnovas 2017-04-18 01:01:27 PDT
(In reply to youenn fablet from comment #7)
> Not sure why they fail.
> You can add specific mac-wk1 [Pass Failure] TestExpectations for these I
> guess.

It seems a timing issue, I've added more info in bug #170935.
I can reproduce the issue now passing "-1" to run-webkit-tests in order to use WK1.
Anyway I'll skip them for the first import and then work on getting them unskipped.

Thanks for the review.
Comment 10 WebKit Commit Bot 2017-04-18 02:17:23 PDT
Comment on attachment 307357 [details]
Patch for landing

Clearing flags on attachment: 307357

Committed r215457: <http://trac.webkit.org/changeset/215457>
Comment 11 WebKit Commit Bot 2017-04-18 02:17:25 PDT
All reviewed patches have been landed.  Closing bug.
Comment 12 Ryan Haddad 2017-04-18 09:33:56 PDT
It looks like the tests should have been marked as 'ImageOnlyFailure' for mac-wk1. I took care of that in https://trac.webkit.org/changeset/215464/webkit.
Comment 13 Manuel Rego Casasnovas 2017-04-18 23:47:12 PDT
(In reply to Ryan Haddad from comment #12)
> It looks like the tests should have been marked as 'ImageOnlyFailure' for
> mac-wk1. I took care of that in
> https://trac.webkit.org/changeset/215464/webkit.

Oops, sorry about that. Thanks for fixing it.