Bug 159712 – Refresh WPT tests up to c875b42

• Status: RESOLVED
• Resolution: FIXED
• Component: Text

[youenn fablet]

Refresh WPT tests up to c875b42

[youenn fablet]

Created attachment 283511 [details]
Patch

[Build Bot]

Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1673762

New failing tests:
imported/w3c/web-platform-tests/html/dom/interfaces.html
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm

[Build Bot]

Created attachment 283512 [details]
Archive of layout-test-results from ews102 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1673760

New failing tests:
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm

[Build Bot]

Created attachment 283513 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/1673808

New failing tests:
imported/w3c/web-platform-tests/html/dom/interfaces.html
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm

[Build Bot]

Created attachment 283515 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 283511 [details]
Patch

Attachment 283511 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/1673827

New failing tests:
imported/w3c/web-platform-tests/XMLHttpRequest/response-method.htm

[Build Bot]

Created attachment 283519 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.5

[youenn fablet]

Created attachment 283520 [details]
Rebasing failing tests

[Chris Dumez]

Comment on attachment 283520 [details]
Rebasing failing tests

View in context: https://bugs.webkit.org/attachment.cgi?id=283520&action=review

r=me

> LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-1.sub-expected.txt:6
> +TIMEOUT Check the incumbent global EventListeners  are called with Test timed out

Isn't this a sub resource (given the .sub in the name)? Should it be skipped instead of having an -expected.txt?

> LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-2.sub-expected.txt:4
> +Harness Error (TIMEOUT), message = null

Isn't this a sub resource (given the .sub in the name)? Should it be skipped instead of having an -expected.txt?

[Chris Dumez]

Comment on attachment 283520 [details]
Rebasing failing tests

Temporarily clearing the cq until you answer my question. cq+ again if my comment is wrong.

[youenn fablet]

(In reply to comment #11)
> Comment on attachment 283520 [details]
> Rebasing failing tests
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=283520&action=review
> 
> r=me
> 
> > LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-1.sub-expected.txt:6
> > +TIMEOUT Check the incumbent global EventListeners  are called with Test timed out
> 
> Isn't this a sub resource (given the .sub in the name)? Should it be skipped
> instead of having an -expected.txt?
> 
> > LayoutTests/imported/w3c/web-platform-tests/dom/events/EventListener-incumbent-global-2.sub-expected.txt:4
> > +Harness Error (TIMEOUT), message = null
> 
> Isn't this a sub resource (given the .sub in the name)? Should it be skipped
> instead of having an -expected.txt?

WPT has some processing rules based on filenames.
Files like xx.sub.html will be preprocessed within wpt server to replace some macros, typically hostname and ports related macros.
Files like yy.https.html will be executed after being loaded through https.

Currently, the test importer is trying to detect whether files are resources or tests by looking at testharness.js/testharnessreport.js links.
Ideally, we should use the scripts in wpt to generate that list but so far, this heuristic is kind of working.

(In reply to comment #12)
> Comment on attachment 283520 [details]
> Rebasing failing tests
> 
> Temporarily clearing the cq until you answer my question. cq+ again if my
> comment is wrong.

np

[youenn fablet]

> > Isn't this a sub resource (given the .sub in the name)? Should it be skipped
> > instead of having an -expected.txt?
> 
> WPT has some processing rules based on filenames.
> Files like xx.sub.html will be preprocessed within wpt server to replace
> some macros, typically hostname and ports related macros.
> Files like yy.https.html will be executed after being loaded through https.

For js files, you may find some links like src="test.js?pipe=sub"
This is the same kind of processing.

[WebKit Commit Bot]

Comment on attachment 283520 [details]
Rebasing failing tests

Clearing flags on attachment: 283520

Committed r203164: <http://trac.webkit.org/changeset/203164>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Michael Catanzaro]

Reopening because several failing tests are marked against this bug:

webkit.org/b/159712 imported/w3c/web-platform-tests/html/semantics/document-metadata/the-link-element/document-without-browsing-context.html [ Timeout ]
webkit.org/b/159712 imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping.html [ Failure Pass ]
webkit.org/b/159712 imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping.html [ Failure Pass ]
webkit.org/b/159712 imported/w3c/web-platform-tests/XMLHttpRequest/open-url-multi-window-6.htm [ Pass Timeout ]

You can mark them against a new bug if you want to close this one!

I'm also updating the expectations for imported/w3c/web-platform-tests/html/semantics/document-metadata/the-link-element/document-without-browsing-context.html to reflect that it's no longer timing out, but now flakily passing/failing, since r203637-r203638 on the GTK port.

[youenn fablet]

Created attachment 290797 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 290797 [details]
Patch

Clearing flags on attachment: 290797

Committed r206860: <http://trac.webkit.org/changeset/206860>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.