Bug 170818 - LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription.html is a flaky failure
Summary: LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemote...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Matt Lewis
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-04-13 12:16 PDT by Ryan Haddad
Modified: 2017-06-01 10:04 PDT (History)
2 users (show)

See Also:


Attachments
Mark test as flaky (1.41 KB, patch)
2017-05-02 11:15 PDT, Matt Lewis
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2017-04-13 12:16:47 PDT
LayoutTest imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r215322%20(501)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Frtcpeerconnection%2FsetRemoteDescription.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/rtcpeerconnection/setRemoteDescription-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Triggers onaddstream when called with a remote description and the MSID is parsed. 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Triggers onaddstream when called with a remote description and the MSID is parsed. Test timed out
+
Comment 1 Matt Lewis 2017-05-02 11:15:45 PDT
Created attachment 308836 [details]
Mark test as flaky
Comment 2 Matt Lewis 2017-05-02 11:16:23 PDT
(In reply to Matt Lewis from comment #1)
> Created attachment 308836 [details]
> Patch

Added to test expectations.
Comment 3 Ryan Haddad 2017-05-02 11:45:22 PDT
Comment on attachment 308836 [details]
Mark test as flaky

Clearing flags on attachment: 308836

Committed r216081: <http://trac.webkit.org/changeset/216081>
Comment 4 Matt Lewis 2017-05-15 16:21:35 PDT
This test is consistently failing with these other tests:
imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCDataChannel-id-actual.txt
@@ -1,4 +1,4 @@
 
-FAIL DTLS client uses odd data channel IDs assert_equals: id expected 1 but got 0
-FAIL DTLS server uses even data channel IDs assert_not_equals: got disallowed value null
+FAIL DTLS client uses odd data channel IDs promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'pc.localDescription.sdp')"
+FAIL DTLS server uses even data channel IDs promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'pc.localDescription.sdp')"
 
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-createDataChannel-actual.txt
@@ -22,7 +22,7 @@
 PASS createDataChannel with id -1 
 PASS createDataChannel with id 0 
 PASS createDataChannel with id 1 
-FAIL createDataChannel with id 65534 The operation is not supported.
+PASS createDataChannel with id 65534 
 PASS createDataChannel with id 65535 
 PASS createDataChannel with id 65536 
 FAIL createDataChannel with priority "high" assert_equals: expected (string) "high" but got (undefined) undefined

imported/w3c/web-platform-tests/webrtc/getstats.html
Diff:
--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/getstats-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/getstats-actual.txt
@@ -1,5 +1,5 @@
 
-FAIL Can get stats from a basic WebRTC call. assert_unreached: Error: assert_not_equals: Did not find peer-connection stats got disallowed value null:  Reached unreachable code
+FAIL Can get stats from a basic WebRTC call. assert_unreached: Error TypeError: Argument 1 ('description') to RTCPeerConnection.setRemoteDescription must be an instance of RTCSessionDescription happened at step Set remote description at second Reached unreachable code
 Retrieved stats info