Bug 171930 – LayoutTest/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-idl.html is flaky

• Component: New Bugs
• Status: NEW

[Matt Lewis]

The test imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-idl.html is a flaky failure on Sierra Debug and is consistently failing on El Capitan and iOS simulator.

test was added in https://trac.webkit.org/changeset/216537/webkit

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2FRTCPeerConnection-idl.html

build:
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/builds/917
https://build.webkit.org/results/Apple%20Sierra%20Debug%20WK2%20(Tests)/r216581%20(917)/results.html

diff:
--- /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-idl-expected.txt
+++ /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-idl-actual.txt
@@ -1,7 +1,16 @@
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Can only call RTCPeerConnection.createOffer on instances of RTCPeerConnection
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Can only call RTCPeerConnection.createAnswer on instances of RTCPeerConnection
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Can only call RTCPeerConnection.addIceCandidate on instances of RTCPeerConnection
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Can only call RTCPeerConnection.getStats on instances of RTCPeerConnection
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Argument 1 ('description') to RTCPeerConnection.setLocalDescription must be an instance of RTCSessionDescription
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Argument 1 ('description') to RTCPeerConnection.setRemoteDescription must be an instance of RTCSessionDescription
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Not enough arguments
 Description
 
 This test verifies the availability of the RTCPeerConnection interface.
 
+
+Harness Error (FAIL), message = Not enough arguments
 
 PASS EventTarget interface: existence and properties of interface object 
 PASS EventTarget interface object length

[Matt Lewis]

Marked test flaky in test expectations.
 https://trac.webkit.org/changeset/216594/webkit/

[Matt Lewis]

While working on another group of test in the same grouping of imported/w3c/web-platform-tests/webrtc/ I have found this test is now failing every time on all WK2 platforms.