We should import XHR related web platform tests (https://github.com/w3c/web-platform-tests/tree/master/XMLHttpRequest)
Created attachment 247676 [details] XHR import based on bug 134766 new tool
Created attachment 254365 [details] Patch
Comment on attachment 254365 [details] Patch Attachment 254365 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.appspot.com/results/5811620366254080 New failing tests: imported/w3c/web-platform-tests/XMLHttpRequest/send-network-error-async-events.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-cors.htm imported/w3c/web-platform-tests/XMLHttpRequest/open-url-worker-origin.htm imported/w3c/web-platform-tests/XMLHttpRequest/event-upload-progress-crossorigin.sub.htm
Created attachment 254369 [details] Archive of layout-test-results from ews104 for mac-mavericks-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews104 Port: mac-mavericks-wk2 Platform: Mac OS X 10.9.5
Comment on attachment 254365 [details] Patch Attachment 254365 [details] did not pass mac-ews (mac): Output: http://webkit-queues.appspot.com/results/4711815606960128 New failing tests: imported/w3c/web-platform-tests/XMLHttpRequest/send-network-error-async-events.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-cors.htm imported/w3c/web-platform-tests/XMLHttpRequest/open-url-worker-origin.htm imported/w3c/web-platform-tests/XMLHttpRequest/event-upload-progress-crossorigin.sub.htm
Created attachment 254372 [details] Archive of layout-test-results from ews101 for mac-mavericks The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews101 Port: mac-mavericks Platform: Mac OS X 10.9.5
Created attachment 263681 [details] Rebasing tests
Comment on attachment 263681 [details] Rebasing tests Attachment 263681 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/316544 New failing tests: imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-network-error-async-events.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-entity-body-basic.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-repeat-no-args.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-setrequestheader-existing-session.htm imported/w3c/web-platform-tests/XMLHttpRequest/event-upload-progress-crossorigin.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/preserve-ua-header-on-redirect.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-setrequestheader.htm
Created attachment 263690 [details] Archive of layout-test-results from ews101 for mac-mavericks The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews101 Port: mac-mavericks Platform: Mac OS X 10.9.5
Comment on attachment 263681 [details] Rebasing tests Attachment 263681 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/316548 New failing tests: imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-network-error-async-events.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-repeat-no-args.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-setrequestheader-existing-session.htm imported/w3c/web-platform-tests/XMLHttpRequest/preserve-ua-header-on-redirect.htm imported/w3c/web-platform-tests/XMLHttpRequest/event-upload-progress-crossorigin.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-authentication-basic-setrequestheader.htm
Created attachment 263692 [details] Archive of layout-test-results from ews107 for mac-mavericks-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews107 Port: mac-mavericks-wk2 Platform: Mac OS X 10.9.5
Comment on attachment 263681 [details] Rebasing tests Attachment 263681 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/316541 New failing tests: imported/w3c/web-platform-tests/XMLHttpRequest/send-network-error-async-events.sub.htm imported/w3c/web-platform-tests/XMLHttpRequest/send-entity-body-basic.htm imported/w3c/web-platform-tests/XMLHttpRequest/event-upload-progress-crossorigin.sub.htm
Created attachment 263693 [details] Archive of layout-test-results from ews112 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews112 Port: mac-yosemite Platform: Mac OS X 10.10.5
Created attachment 263694 [details] Adding additional Failure expectations
Created attachment 263704 [details] Adding additional Failure expectations
Created attachment 263723 [details] Skipping import of failing tests
Comment on attachment 263723 [details] Skipping import of failing tests View in context: https://bugs.webkit.org/attachment.cgi?id=263723&action=review > LayoutTests/imported/w3c/ChangeLog:8 > + * resources/ImportExpectations: skipping import of some XHR tests that are failing or timing out. Why not skip running them instead of skip importing them? > LayoutTests/TestExpectations:354 > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-aborted.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-abortedonmain.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-overrides.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-overridesexpires.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-simple.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-synconmain.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-twice.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-aborted.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-overrides.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-overridesexpires.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-simple.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-synconworker.html [ Slow ] > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-twice.html [ Slow ] This seems like a problem. Is there any trickier way we can test timeouts without slowing down our testing like this? Long term it’s not good to let these start creeping in.
(In reply to comment #17) > Comment on attachment 263723 [details] > Skipping import of failing tests > > View in context: > https://bugs.webkit.org/attachment.cgi?id=263723&action=review > > > LayoutTests/imported/w3c/ChangeLog:8 > > + * resources/ImportExpectations: skipping import of some XHR tests that are failing or timing out. > > Why not skip running them instead of skip importing them? It could be done that way too. These bugs are failing for various reasons. For instance, some of them use URLs like www2.localhost which should be ok and served as 127.0.0.1 but are blocked by WTR (and probably unresolvable too). Some tests are working in some environments but not others. I thought it would be nicer to import them (as [ Skip ] or not) with proper bugs attached to them. > > > LayoutTests/TestExpectations:354 > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-aborted.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-abortedonmain.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-overrides.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-overridesexpires.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-simple.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-synconmain.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-twice.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-aborted.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-overrides.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-overridesexpires.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-simple.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-synconworker.html [ Slow ] > > +imported/w3c/web-platform-tests/XMLHttpRequest/xmlhttprequest-timeout-worker-twice.html [ Slow ] > > This seems like a problem. Is there any trickier way we can test timeouts > without slowing down our testing like this? Long term it’s not good to let > these start creeping in. These tests are marked as slow (<meta name=timeout content=long>). In the WPT framework, testharness.js will check this meta tag, and would use a long timeout where necessary. In current WebKit test framework, we cannot do otherwise than marking these tests as Slow in TestExpectations, at least at first import to reduce the risk of flakiness. Once they will be run several times, we may be able to remove [ Slow ] for some bots. Also, these tests are updated versions of LayoutTests/http/tests/xmlhttprequest/timeout/ which are often flaky currently on WebKit bots. One reason may be that these tests use various XHR timeout values between 200ms and 1s. WPT tests increased some XHR timeout values to 5s, which is close to some bots regular test timeout.
Comment on attachment 263723 [details] Skipping import of failing tests Clearing flags on attachment: 263723 Committed r191546: <http://trac.webkit.org/changeset/191546>
All reviewed patches have been landed. Closing bug.
These tests have uncovered a lot of leaks, jumping from ~600Kb to ~2Mb: https://build.webkit.org/builders/Apple%20El%20Capitan%20%28Leaks%29/builds/593 https://build.webkit.org/builders/Apple%20El%20Capitan%20%28Leaks%29/builds/594 We really need to find a way to make the leak bot more precise, but this case is pretty clear.
Filed bug 150546 about that.
Several of these tests fail on Windows: https://build.webkit.org/results/Apple%20Win%207%20Debug%20(Tests)/r191622%20(68088)/results.html