Bug 214286 - [ Mac WK2 & iOS ] imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-connectionState.https.html is flaky
Summary: [ Mac WK2 & iOS ] imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-co...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-07-13 16:19 PDT by Truitt Savell
Modified: 2021-09-10 09:16 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Truitt Savell 2020-07-13 16:19:37 PDT
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-connectionState.https.html

This test has been flaky sense introduction in https://trac.webkit.org/changeset/264202/webkit

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2FRTCPeerConnection-connectionState.https.html

Diff:
--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-connectionState.https-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-connectionState.https-actual.txt
@@ -3,7 +3,7 @@
 PASS Closing the connection should set connectionState to closed 
 PASS connection with one data channel should eventually have connected connection state 
 FAIL connection with one data channel should eventually have transports in connected state undefined is not an object (evaluating 'sctpTransport.transport')
-FAIL connectionState remains new when not adding remote ice candidates assert_equals: expected "new" but got "checking"
+FAIL connectionState remains new when not adding remote ice candidates assert_equals: expected "new" but got "completed"
 PASS connectionState transitions to connected via connecting 
 PASS Closing a PeerConnection should not fire connectionstatechange event 
 

I was able to repro a failure but it was a different diff result. I reproduced using run-webkit-tests imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-connectionState.https.html --iterations 1000 -f --exit-after-n-failures 1
Comment 1 Radar WebKit Bug Importer 2020-07-13 16:20:07 PDT
<rdar://problem/65507996>
Comment 2 Truitt Savell 2020-07-13 16:31:35 PDT
Marked this as flaky in https://trac.webkit.org/changeset/264325/webkit while this is investigated.
Comment 3 ayumi_kojima 2021-09-10 09:16:03 PDT
The test is failing on iOS as well. Updated test expectations: https://trac.webkit.org/changeset/282272/webkit