Bug 142152

Summary: media/track/track-in-band-cues-added-once.html flakily fails
Product: WebKit Reporter: Alexey Proskuryakov <ap>
Component: MediaAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: darin, eric.carlson, jer.noble, jlewis3, ryanhaddad
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   

Description Alexey Proskuryakov 2015-03-01 12:37:24 PST
media/track/track-in-band-cues-added-once.html frequently fails on bots:

@@ -18,7 +18,6 @@
 EVENT(change)
 RUN(video.play())
 
-RUN(video.pause())
 RUN(video.play())
 
 RUN(video.pause())

http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Ftrack%2Ftrack-in-band-cues-added-once.html
Comment 1 Alexey Proskuryakov 2015-03-01 12:38:43 PST
I can reproduce locally with:

run-webkit-tests --no-retry -2 media/track/track-in-band-cues-added-once.html --repeat 100 -f
Comment 2 Alexey Proskuryakov 2015-03-01 12:46:24 PST
The test is quite complex, I'm not sure if this failure discovers a real bug, or a test shortcoming.

Marked as flaky in http://trac.webkit.org/r180868.
Comment 3 Alexey Proskuryakov 2015-03-02 10:16:18 PST
A different type of failure:

@@ -18,6 +18,8 @@
 EVENT(change)
 RUN(video.play())
 
+RUN(video.play())
+
 RUN(video.pause())
 RUN(video.play())
Comment 4 Ryan Haddad 2016-08-03 18:29:00 PDT
(In reply to comment #3)
> A different type of failure:
> 
> @@ -18,6 +18,8 @@
>  EVENT(change)
>  RUN(video.play())
>  
> +RUN(video.play())
> +
>  RUN(video.pause())
>  RUN(video.play())

The test was recently re-enabled on iOS, and it mirrors this flaky failure https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Ftrack%2Ftrack-in-band-cues-added-once.html

Marked as flaky on ios-simulator-wk2 in http://trac.webkit.org/projects/webkit/changeset/204109
Comment 5 Matt Lewis 2017-06-06 17:55:39 PDT
The test is now starting to Time out according to the Flakiness Dashboard on all macOS Debug.

Diff:
--- /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/media/track/track-in-band-cues-added-once-expected.txt
+++ /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/media/track/track-in-band-cues-added-once-actual.txt
@@ -1,26 +1,5 @@
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 48403)
+FAIL: Timed out waiting for notifyDone to be called
 
-In-band text tracks.
-
-EVENT(addtrack)
-EXPECTED (event.track == 'video.textTracks[0]') OK
-
-EVENT(canplaythrough)
-
-** Check initial in-band track states
-EXPECTED (video.textTracks.length == '1') OK
-RUN(inbandTrack1 = video.textTracks[0])
-EXPECTED (inbandTrack1.mode == 'disabled') OK
-EXPECTED (inbandTrack1.cues == 'null') OK
-EXPECTED (inbandTrack1.language == 'en') OK
-EXPECTED (inbandTrack1.kind == 'captions') OK
-
-RUN(inbandTrack1.mode = 'showing')
-EVENT(change)
-RUN(video.play())
-
-RUN(video.pause())
-RUN(video.play())
-
-RUN(video.pause())
-END OF TEST
-
+#EOF
+#EOF
Comment 6 Matt Lewis 2017-06-16 13:42:42 PDT
According to the flakiness dashboard, this test has not passed as far back as the data goes. The expectations were set to [ Pass Failure ]. I adjusted the test expectations in r218410 https://trac.webkit.org/changeset/218410/webkit
Comment 7 Darin Adler 2019-12-20 09:51:36 PST
media/track/track-in-band-cues-added-once.html is passing on my Catalina machine, not failing nor timing out
Comment 8 Alexey Proskuryakov 2019-12-20 10:05:36 PST
It's still failing a lot on bots: https://results.webkit.org/?suite=layout-tests&test=media%2Ftrack%2Ftrack-in-band-cues-added-once.html
Comment 9 Darin Adler 2019-12-20 12:55:26 PST
That bot result looks like [ Pass Failure ].

The TestExpectations file says [ Failure Timeout ].

My comment wasn’t an effort to get "Failure" removed.
Comment 10 Alexey Proskuryakov 2019-12-20 14:52:55 PST
Updated TestExpectations in http://trac.webkit.org/r253846
Comment 11 Darin Adler 2019-12-20 14:55:19 PST
Thanks, that's great. I would have done it myself if I was a little more sure.