Bug 170818 – LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription.html is a flaky failure

• Status: NEW
• Component: New Bugs

[Ryan Haddad]

LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r215322%20(501)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Frtcpeerconnection%2FsetRemoteDescription.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Triggers onaddstream when called with a remote description and the MSID is parsed. 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Triggers onaddstream when called with a remote description and the MSID is parsed. Test timed out
+

[Matt Lewis]

Created attachment 308836 [details]
Mark test as flaky

[Matt Lewis]

(In reply to Matt Lewis from comment #1)
> Created attachment 308836 [details]
> Patch

Added to test expectations.

[Ryan Haddad]

Comment on attachment 308836 [details]
Mark test as flaky

Clearing flags on attachment: 308836

Committed r216081: <http://trac.webkit.org/changeset/216081>

[Matt Lewis]

This test is consistently failing with these other tests:
imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id-actual.txt
@@ -1,4 +1,4 @@
 
-FAIL DTLS client uses odd data channel IDs assert_equals: id expected 1 but got 0
-FAIL DTLS server uses even data channel IDs assert_not_equals: got disallowed value null
+FAIL DTLS client uses odd data channel IDs promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'pc.localDescription.sdp')"
+FAIL DTLS server uses even data channel IDs promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'pc.localDescription.sdp')"
 
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel-actual.txt
@@ -22,7 +22,7 @@
 PASS createDataChannel with id -1 
 PASS createDataChannel with id 0 
 PASS createDataChannel with id 1 
-FAIL createDataChannel with id 65534 The operation is not supported.
+PASS createDataChannel with id 65534 
 PASS createDataChannel with id 65535 
 PASS createDataChannel with id 65536 
 FAIL createDataChannel with priority "high" assert_equals: expected (string) "high" but got (undefined) undefined

imported/w3c/web-platform-tests/webrtc/getstats.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/getstats-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/getstats-actual.txt
@@ -1,5 +1,5 @@
 
-FAIL Can get stats from a basic WebRTC call. assert_unreached: Error: assert_not_equals: Did not find peer-connection stats got disallowed value null:  Reached unreachable code
+FAIL Can get stats from a basic WebRTC call. assert_unreached: Error TypeError: Argument 1 ('description') to RTCPeerConnection.setRemoteDescription must be an instance of RTCSessionDescription happened at step Set remote description at second Reached unreachable code
 Retrieved stats info