171895 2017-05-09 16:22:42 -0700 [wk2] Layout Test/webrtc/audio-replace-track.html is flaky 2017-06-02 17:12:20 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=171815 https://bugs.webkit.org/show_bug.cgi?id=171292 https://bugs.webkit.org/show_bug.cgi?id=172645 InRadar P2 Normal --- 1 jlewis3 webkit-unassigned ryanhaddad webkit-bug-importer youennf oldest_to_newest 1306660 0 jlewis3 2017-05-09 16:22:42 -0700 the test webrtc/audio-replace-track.html is a flaky failure and timeout. https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webrtc%2Faudio-replace-track.html build: https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/builds/902 https://build.webkit.org/results/Apple%20Sierra%20Debug%20WK2%20(Tests)/r216534%20(902)/results.html Diff: --- /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/webrtc/audio-replace-track-expected.txt +++ /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/webrtc/audio-replace-track-actual.txt @@ -1,3 +1,3 @@ -PASS Replacing audio track from a peer connection +FAIL Replacing audio track from a peer connection promise_test: Unhandled rejection with value: object "TypeError: null is not an object (evaluating 'context.createMediaStreamSource')" 1306664 1 jlewis3 2017-05-09 16:28:28 -0700 Marked test as flaky https://trac.webkit.org/changeset/216557/webkit 1314280 2 jlewis3 2017-05-31 13:01:01 -0700 The test has now started to become flaky on iOS as well. Added to see also a bug that looks to be an attempt to help with flakiness. Oldest iOS failure on dashboard is on 5/22 with this build: https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Release%20WK2%20(Tests)/builds/1677 additional diff: --- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/webrtc/audio-replace-track-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/webrtc/audio-replace-track-actual.txt @@ -1,6 +1,6 @@ PASS Starting an audio connection -PASS Ensuring mock audio source is received on the remote side +FAIL Ensuring mock audio source is received on the remote side assert_true: heard hum 1 expected true got false PASS Using replaceTrack for audio -PASS Ensuring remote audio gets the replacing track data +FAIL Ensuring remote audio gets the replacing track data assert_true: heard hum 2 expected true got false 1315268 3 ryanhaddad 2017-06-02 16:08:24 -0700 Marked test as flaky on iOS and removed timeout expectation for macOS in https://trac.webkit.org/r217739 1315304 4 webkit-bug-importer 2017-06-02 17:12:20 -0700 <rdar://problem/32548741>