Bug 159712

Summary: Refresh WPT tests up to c875b42
Product: WebKit Reporter: youenn fablet <youennf>
Component: TextAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, buildbot, cdumez, commit-queue, mcatanzaro, mmaxfield, rniwa
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews102 for mac-yosemite
none
Archive of layout-test-results from ews114 for mac-yosemite
none
Archive of layout-test-results from ews104 for mac-yosemite-wk2
none
Archive of layout-test-results from ews122 for ios-simulator-wk2
none
Rebasing failing tests
none
Patch none

Description youenn fablet 2016-07-13 04:05:30 PDT
Refresh WPT tests up to c875b42
Comment 1 youenn fablet 2016-07-13 04:08:52 PDT
Created attachment 283511 [details]
Patch
Comment 2 Build Bot 2016-07-13 04:46:26 PDT
Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1673762

New failing tests:
imported/w3c/web-platform-tests/html/dom/interfaces.html
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm
Comment 3 Build Bot 2016-07-13 04:46:29 PDT
Created attachment 283512 [details]
Archive of layout-test-results from ews102 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 4 Build Bot 2016-07-13 04:52:35 PDT
Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1673760

New failing tests:
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm
Comment 5 Build Bot 2016-07-13 04:52:37 PDT
Created attachment 283513 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 6 Build Bot 2016-07-13 05:00:05 PDT
Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/1673808

New failing tests:
imported/w3c/web-platform-tests/html/dom/interfaces.html
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm
Comment 7 Build Bot 2016-07-13 05:00:08 PDT
Created attachment 283515 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5
Comment 8 Build Bot 2016-07-13 05:19:46 PDT
Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/1673827

New failing tests:
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm
Comment 9 Build Bot 2016-07-13 05:19:49 PDT
Created attachment 283519 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.5
Comment 10 youenn fablet 2016-07-13 05:26:39 PDT
Created attachment 283520 [details]
Rebasing failing tests
Comment 11 Chris Dumez 2016-07-13 09:07:36 PDT
Comment on attachment 283520 [details]
Rebasing failing tests

View in context: https://bugs.webkit.org/attachment.cgi?id=283520&action=review

r=me

> LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-1.sub-expected.txt:6
> +TIMEOUT Check the incumbent global EventListeners  are called with Test timed out

Isn't this a sub resource (given the .sub in the name)? Should it be skipped instead of having an -expected.txt?

> LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-2.sub-expected.txt:4
> +Harness Error (TIMEOUT), message = null

Isn't this a sub resource (given the .sub in the name)? Should it be skipped instead of having an -expected.txt?
Comment 12 Chris Dumez 2016-07-13 09:08:12 PDT
Comment on attachment 283520 [details]
Rebasing failing tests

Temporarily clearing the cq until you answer my question. cq+ again if my comment is wrong.
Comment 13 youenn fablet 2016-07-13 09:16:38 PDT
(In reply to comment #11)
> Comment on attachment 283520 [details]
> Rebasing failing tests
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=283520&action=review
> 
> r=me
> 
> > LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-1.sub-expected.txt:6
> > +TIMEOUT Check the incumbent global EventListeners  are called with Test timed out
> 
> Isn't this a sub resource (given the .sub in the name)? Should it be skipped
> instead of having an -expected.txt?
> 
> > LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-2.sub-expected.txt:4
> > +Harness Error (TIMEOUT), message = null
> 
> Isn't this a sub resource (given the .sub in the name)? Should it be skipped
> instead of having an -expected.txt?

WPT has some processing rules based on filenames.
Files like xx.sub.html will be preprocessed within wpt server to replace some macros, typically hostname and ports related macros.
Files like yy.https.html will be executed after being loaded through https.

Currently, the test importer is trying to detect whether files are resources or tests by looking at testharness.js/testharnessreport.js links.
Ideally, we should use the scripts in wpt to generate that list but so far, this heuristic is kind of working.

(In reply to comment #12)
> Comment on attachment 283520 [details]
> Rebasing failing tests
> 
> Temporarily clearing the cq until you answer my question. cq+ again if my
> comment is wrong.

np
Comment 14 youenn fablet 2016-07-13 09:18:03 PDT
> > Isn't this a sub resource (given the .sub in the name)? Should it be skipped
> > instead of having an -expected.txt?
> 
> WPT has some processing rules based on filenames.
> Files like xx.sub.html will be preprocessed within wpt server to replace
> some macros, typically hostname and ports related macros.
> Files like yy.https.html will be executed after being loaded through https.

For js files, you may find some links like src="test.js?pipe=sub"
This is the same kind of processing.
Comment 15 WebKit Commit Bot 2016-07-13 09:31:27 PDT
Comment on attachment 283520 [details]
Rebasing failing tests

Clearing flags on attachment: 283520

Committed r203164: <http://trac.webkit.org/changeset/203164>
Comment 16 WebKit Commit Bot 2016-07-13 09:31:32 PDT
All reviewed patches have been landed.  Closing bug.
Comment 17 Michael Catanzaro 2016-07-23 15:08:43 PDT
Reopening because several failing tests are marked against this bug:

webkit.org/b/159712 imported/w3c/web-platform-tests/html/semantics/document-metadata/the-link-element/document-without-browsing-context.html [ Timeout ]
webkit.org/b/159712 imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping.html [ Failure Pass ]
webkit.org/b/159712 imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping.html [ Failure Pass ]
webkit.org/b/159712 imported/w3c/web-platform-tests/XMLHttpRequest/open-url-multi-window-6.htm [ Pass Timeout ]

You can mark them against a new bug if you want to close this one!

I'm also updating the expectations for imported/w3c/web-platform-tests/html/semantics/document-metadata/the-link-element/document-without-browsing-context.html to reflect that it's no longer timing out, but now flakily passing/failing, since r203637-r203638 on the GTK port.
Comment 18 youenn fablet 2016-10-06 02:07:50 PDT
Created attachment 290797 [details]
Patch
Comment 19 WebKit Commit Bot 2016-10-06 03:40:43 PDT
Comment on attachment 290797 [details]
Patch

Clearing flags on attachment: 290797

Committed r206860: <http://trac.webkit.org/changeset/206860>
Comment 20 WebKit Commit Bot 2016-10-06 03:40:50 PDT
All reviewed patches have been landed.  Closing bug.