Bug 170818

Summary: LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription.html is a flaky failure
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: New BugsAssignee: Matt Lewis <jlewis3>
Status: NEW ---    
Severity: Normal CC: jlewis3, youennf
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=172521
https://bugs.webkit.org/show_bug.cgi?id=172821
Attachments:
Description Flags
Mark test as flaky none

Description Ryan Haddad 2017-04-13 12:16:47 PDT
LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r215322%20(501)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Frtcpeerconnection%2FsetRemoteDescription.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Triggers onaddstream when called with a remote description and the MSID is parsed. 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Triggers onaddstream when called with a remote description and the MSID is parsed. Test timed out
+
Comment 1 Matt Lewis 2017-05-02 11:15:45 PDT
Created attachment 308836 [details]
Mark test as flaky
Comment 2 Matt Lewis 2017-05-02 11:16:23 PDT
(In reply to Matt Lewis from comment #1)
> Created attachment 308836 [details]
> Patch

Added to test expectations.
Comment 3 Ryan Haddad 2017-05-02 11:45:22 PDT
Comment on attachment 308836 [details]
Mark test as flaky

Clearing flags on attachment: 308836

Committed r216081: <http://trac.webkit.org/changeset/216081>
Comment 4 Matt Lewis 2017-05-15 16:21:35 PDT
This test is consistently failing with these other tests:
imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id-actual.txt
@@ -1,4 +1,4 @@
 
-FAIL DTLS client uses odd data channel IDs assert_equals: id expected 1 but got 0
-FAIL DTLS server uses even data channel IDs assert_not_equals: got disallowed value null
+FAIL DTLS client uses odd data channel IDs promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'pc.localDescription.sdp')"
+FAIL DTLS server uses even data channel IDs promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'pc.localDescription.sdp')"
 
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel-actual.txt
@@ -22,7 +22,7 @@
 PASS createDataChannel with id -1 
 PASS createDataChannel with id 0 
 PASS createDataChannel with id 1 
-FAIL createDataChannel with id 65534 The operation is not supported.
+PASS createDataChannel with id 65534 
 PASS createDataChannel with id 65535 
 PASS createDataChannel with id 65536 
 FAIL createDataChannel with priority "high" assert_equals: expected (string) "high" but got (undefined) undefined

imported/w3c/web-platform-tests/webrtc/getstats.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/getstats-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/getstats-actual.txt
@@ -1,5 +1,5 @@
 
-FAIL Can get stats from a basic WebRTC call. assert_unreached: Error: assert_not_equals: Did not find peer-connection stats got disallowed value null:  Reached unreachable code
+FAIL Can get stats from a basic WebRTC call. assert_unreached: Error TypeError: Argument 1 ('description') to RTCPeerConnection.setRemoteDescription must be an instance of RTCSessionDescription happened at step Set remote description at second Reached unreachable code
 Retrieved stats info