Bug 177501

Summary: Layout Test webrtc/video-mute.html is flaky.
Product: WebKit Reporter: Matt Lewis <jlewis3>
Component: WebRTCAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: commit-queue, eric.carlson, ews-watchlist, ryanhaddad, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews126 for ios-simulator-wk2
none
Patch
none
Patch none

Description Matt Lewis 2017-09-26 13:12:06 PDT
webrtc/video-mute.html is a flaky timeout on iOS 11 Release WK2 according to the flakiness dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webrtc%2Fvideo-mute.html

build:
https://build.webkit.org/results/Apple%20iOS%2011%20Simulator%20Release%20WK2%20(Tests)/r222503%20(118)/results.html

Diff:

--- /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/webrtc/video-mute-expected.txt
+++ /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/webrtc/video-mute-actual.txt
@@ -1,11 +1,5 @@
-Video should be running, go to black and running.
-Following, should be a snapshot of the video, a black frame and a snapshot of the video.
-  
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 25137)
+FAIL: Timed out waiting for notifyDone to be called
 
-PASS Setting video exchange 
-PASS Ensuring connection state is connected 
-PASS Track is enabled, video should not be black 
-PASS Track is disabled, video should be black 
-PASS If disabled, black frames should still be coming 
-PASS Track is enabled, video should not be black 
-
+#EOF
+#EOF
Comment 1 Matt Lewis 2017-10-26 11:58:06 PDT
Marked the test as flaky on iOS and High Sierra.

https://trac.webkit.org/changeset/224036/webkit
Comment 2 Ryan Haddad 2018-01-19 16:17:02 PST
This also happens on Sierra. Updated TestExpectations in https://trac.webkit.org/r227250.
Comment 3 Matt Lewis 2018-09-17 17:03:11 PDT
This has started failing consistently on iOS 11 Simulator but has since started to pass on macOS:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webrtc%2Fvideo-mute.html

Strangely, There is no diff for the test.
Comment 4 Matt Lewis 2018-09-17 17:03:29 PDT
Refiling bug for iOS failure.
Comment 5 youenn fablet 2018-09-18 09:48:34 PDT
Created attachment 350025 [details]
Patch
Comment 6 EWS Watchlist 2018-09-18 11:38:01 PDT
Comment on attachment 350025 [details]
Patch

Attachment 350025 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/9258357

New failing tests:
webrtc/video-mute.html
Comment 7 EWS Watchlist 2018-09-18 11:38:03 PDT
Created attachment 350035 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 8 youenn fablet 2018-09-18 15:12:00 PDT
Created attachment 350065 [details]
Patch
Comment 9 youenn fablet 2018-09-18 15:36:18 PDT
Created attachment 350067 [details]
Patch
Comment 10 WebKit Commit Bot 2018-09-19 08:30:13 PDT
Comment on attachment 350067 [details]
Patch

Rejecting attachment 350067 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'validate-changelog', '--check-oops', '--non-interactive', 350067, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

ChangeLog entry in LayoutTests/ChangeLog contains OOPS!.

Full output: https://webkit-queues.webkit.org/results/9270187
Comment 11 WebKit Commit Bot 2018-09-19 10:17:42 PDT
Comment on attachment 350067 [details]
Patch

Clearing flags on attachment: 350067

Committed r236211: <https://trac.webkit.org/changeset/236211>
Comment 12 WebKit Commit Bot 2018-09-19 10:17:43 PDT
All reviewed patches have been landed.  Closing bug.
Comment 13 Radar WebKit Bug Importer 2018-09-19 10:18:25 PDT
<rdar://problem/44606920>