imported/w3c/web-platform-tests/websockets/Create-Secure-verify-url-set-non-default-port.any.html flakily failing since import in 246406 Confirmed issue is not exclusive to WK2 locally with MacOS WK1 repro steps listed below. Steps to Reproduce: MacOS WK1 with Build 246585: run-webkit-tests --iter 1000 -f --exit-after-n-failures 20 imported/w3c/web-platform-tests/websockets/Create-Secure-verify-url-set-non-default-port.any.html -1 iOS-Simulator WK2 with Build 246585: run-webkit-tests --iter 1000 --child-processes 1 --exit-after-n-failures 20 --ios-simulator imported/w3c/web-platform-tests/websockets/Create-Secure-verify-url-set-non-default-port.any.html Flakiness Dashboard: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebsockets%2FCreate-Secure-verify-url-set-non-default-port.any.html
Created attachment 372475 [details] Update Test Expectations
Comment on attachment 372475 [details] Update Test Expectations Clearing flags on attachment: 372475 Committed r246594: <https://trac.webkit.org/changeset/246594>
<rdar://problem/51902173>
Identified additional tests introduced in https://trac.webkit.org/changeset/246406/webkit that have been flakey. Flakey Tests: imported/w3c/web-platform-tests/websockets/close-invalid.any.html imported/w3c/web-platform-tests/websockets/Create-Secure-verify-url-set-non-default-port.any.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-replace.html imported/w3c/web-platform-tests/websockets/constructor/017.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/send/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/send/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/002.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/readyState/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/send/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/002.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/006.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/007.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/bufferedAmount/bufferedAmount-initial.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/bufferedAmount/bufferedAmount-readonly.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/bufferedAmount/bufferedAmount-initial.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/008.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/007.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/008.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/009.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/002.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/006.html
Best might be to skip for now the newly imported tests that are flaky, at least the WSS (secure web socket ones) which seem to be difficult for us to pass since quite a bit of time.
Created attachment 372499 [details] Update TestExpectations
Comment on attachment 372499 [details] Update TestExpectations Clearing flags on attachment: 372499 Committed r246614: <https://trac.webkit.org/changeset/246614>
Cleaned up list of flakey tests by removing duplicate entries. Updated List: imported/w3c/web-platform-tests/websockets/close-invalid.any.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-replace.html imported/w3c/web-platform-tests/websockets/constructor/017.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/send/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/send/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/002.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/url/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/readyState/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/send/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/002.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/006.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/bufferedAmount/bufferedAmount-readonly.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/bufferedAmount/bufferedAmount-initial.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/003.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/007.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/008.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/009.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/004.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/002.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/001.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/constants/004.html
Marked test list as flakey in https://trac.webkit.org/changeset/246614/webkit.
Created attachment 382811 [details] Patch
Created attachment 382826 [details] Patch
Ping review?
Ping review
Comment on attachment 382826 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=382826&action=review > LayoutTests/TestExpectations:3607 > +imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-replace.html [ DumpJSConsoleLogInStdErr ] Why not just the whole directory?
(In reply to Alex Christensen from comment #14) > Comment on attachment 382826 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=382826&action=review > > > LayoutTests/TestExpectations:3607 > > +imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-replace.html [ DumpJSConsoleLogInStdErr ] > > Why not just the whole directory? Yep, we should probably do that for fetch, xhr, service worker and web sockets. Or maybe the whole WPT folder.
Comment on attachment 382826 [details] Patch Clearing flags on attachment: 382826 Committed r252429: <https://trac.webkit.org/changeset/252429>
All reviewed patches have been landed. Closing bug.
imported/w3c/web-platform-tests/websockets/Create-Secure-verify-url-set-non-default-port.any.htm no longer exists, so I'll remove it from test expectations.