Bug 176078 – Layout Test imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-onnegotiationneeded.html is flaky.

• Component: WebRTC
• Status: RESOLVED DUPLICATE

[Matt Lewis]

imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-onnegotiationneeded.html is a flaky failure on Sierra WK2 Release according to the flakiness dashboard.

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2FRTCPeerConnection-onnegotiationneeded.html

Build:
https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r221309%20(3921)/results.html
https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20(Tests)/builds/3921

diff:
--- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-onnegotiationneeded-expected.txt
+++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-onnegotiationneeded-actual.txt
@@ -1,11 +1,11 @@
 
 Harness Error (TIMEOUT), message = null
 
-PASS Creating first data channel should fire negotiationneeded event 
-PASS task for negotiationneeded event should be enqueued for next tick 
+TIMEOUT Creating first data channel should fire negotiationneeded event Test timed out
+NOTRUN task for negotiationneeded event should be enqueued for next tick 
 PASS negotiationneeded event should not fire if connection is closed 
 PASS calling createDataChannel twice should fire negotiationneeded event once 
-TIMEOUT addTransceiver() should fire negotiationneeded event Test timed out
+NOTRUN addTransceiver() should fire negotiationneeded event 
 PASS Calling addTransceiver() twice should fire negotiationneeded event once 
 PASS Calling both addTransceiver() and createDataChannel() should fire negotiationneeded event once 
 PASS negotiationneeded event should not fire if signaling state is not stable 


This test has been failing with other imported/w3c/web-platform-tests/webrtc tests but many failing runs are now just the singular test failing. previous bug in see also.

[Matt Lewis]

Marked as flaky

https://trac.webkit.org/changeset/225348/webkit

[Dawei Fenton (:realdawei)]

this test is now flaky on iOS 

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2FRTCPeerConnection-onnegotiationneeded.html

diff:
--- /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-onnegotiationneeded-expected.txt
+++ /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-onnegotiationneeded-actual.txt
@@ -1,11 +1,11 @@
 
 Harness Error (TIMEOUT), message = null
 
-PASS Creating first data channel should fire negotiationneeded event 
+TIMEOUT Creating first data channel should fire negotiationneeded event Test timed out
 PASS calling createDataChannel twice should fire negotiationneeded event once 
-PASS addTransceiver() should fire negotiationneeded event 
-FAIL Calling addTransceiver() twice should fire negotiationneeded event once assert_unreached: Pending promise should never be resolved. Instead it is fulfilled with: [object Object] Reached unreachable code
-FAIL Calling both addTransceiver() and createDataChannel() should fire negotiationneeded event once assert_unreached: Pending promise should never be resolved. Instead it is fulfilled with: [object Object] Reached unreachable code
+NOTRUN addTransceiver() should fire negotiationneeded event 
+PASS Calling addTransceiver() twice should fire negotiationneeded event once 
+PASS Calling both addTransceiver() and createDataChannel() should fire negotiationneeded event once 
 PASS negotiationneeded event should not fire if signaling state is not stable 
-TIMEOUT negotiationneeded event should fire only after signaling state go back to stable Test timed out
+NOTRUN negotiationneeded event should fire only after signaling state go back to stable

[Dawei Fenton (:realdawei)]

Marked as flaky in iOS

https://trac.webkit.org/changeset/236225/webkit

[youenn fablet]

*** This bug has been marked as a duplicate of bug 231223 ***