The Test imported/w3c/web-platform-tests/webrtc/no-media-call.html is a flaky failure on Mac-wk2 The test consistently fails at the same time as: imported/w3c/web-platform-tests/webrtc/promises-call.html imported/w3c/web-platform-tests/webrtc/datachannel-emptystring.html History: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Fno-media-call.html Build log: https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r216117%20(1135)/results.html
*** Bug 170820 has been marked as a duplicate of this bug. ***
Created attachment 308922 [details] Mark test as flaky.
Comment on attachment 308922 [details] Mark test as flaky. Clearing flags on attachment: 308922 Committed r216125: <http://trac.webkit.org/changeset/216125>
<rdar://problem/31977318>
As of 5/22 and revision https://trac.webkit.org/changeset/217225/webkit the tests above are now failing on iOS Simulator release and Sierra release consistently. Diff for imported/w3c/web-platform-tests/webrtc/promises-call.html: --- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/promises-call-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/promises-call-actual.txt @@ -1,6 +1,8 @@ -PASS Can set up a basic WebRTC call with only data using promises. +Harness Error (TIMEOUT), message = null + +TIMEOUT Can set up a basic WebRTC call with only data using promises. Test timed out iceConnectionState info -First: completed -Second: connected +First: checking +Second: new Diff for imported/w3c/web-platform-tests/webrtc/datachannel-emptystring.html: --- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring-actual.txt @@ -1,8 +1,7 @@ -FAIL Can send empty strings across a WebRTC data channel. assert_equals: Empty message not received expected "" but got "done" +Harness Error (TIMEOUT), message = null + +TIMEOUT Can send empty strings across a WebRTC data channel. Test timed out Messages exchanged -Sent: [empty string] -Sent: "done" -Received: "done"
(In reply to Matt Lewis from comment #5) > As of 5/22 and revision https://trac.webkit.org/changeset/217225/webkit > > the tests above are now failing on iOS Simulator release and Sierra release > consistently. > > Diff for imported/w3c/web-platform-tests/webrtc/promises-call.html: > > --- > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/promises-call-expected.txt > +++ > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/promises-call-actual.txt > @@ -1,6 +1,8 @@ > > -PASS Can set up a basic WebRTC call with only data using promises. > +Harness Error (TIMEOUT), message = null > + > +TIMEOUT Can set up a basic WebRTC call with only data using promises. Test > timed out > iceConnectionState info > > -First: completed > -Second: connected > +First: checking > +Second: new > > Diff for imported/w3c/web-platform-tests/webrtc/datachannel-emptystring.html: > > --- > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring- > expected.txt > +++ > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring- > actual.txt > @@ -1,8 +1,7 @@ > > -FAIL Can send empty strings across a WebRTC data channel. assert_equals: > Empty message not received expected "" but got "done" > +Harness Error (TIMEOUT), message = null > + > +TIMEOUT Can send empty strings across a WebRTC data channel. Test timed out > Messages exchanged > > -Sent: [empty string] > -Sent: "done" > -Received: "done" Added iOS TestExpectations in https://trac.webkit.org/changeset/217257/webkit.