Bug 176929 – Layout Test imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks.html is flaky

• Status: RESOLVED
• Resolution: FIXED
• Component: Media

[Matt Lewis]

imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks.html is flaky on El Capitan WK2 according to the flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Fready-states%2Fautoplay-with-slow-text-tracks.html

current failure:
https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/r222025%20(4461)/results.html
https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/4461

diff:

--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks-actual.txt
@@ -1,4 +1,3 @@
-CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
 CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
 CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
 CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')

[Matt Lewis]

This has since started to become flaky on High Sierra as well

https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK1%20(Tests)/r225573%20(1824)/results.html
https://build.webkit.org/builders/Apple%20High%20Sierra%20Release%20WK1%20(Tests)/builds/1824

[Truitt Savell]

This test has been flakey on Mojave. It seems to not be a flakey failure on many of the other platforms anymore. The diff is the same as it was previously. test is extremely flaky.

[Ryan Haddad]

Marked test as flaky in https://trac.webkit.org/r237353

[youenn fablet]

Created attachment 385970 [details]
Mark test as failing in all platforms

[WebKit Commit Bot]

Comment on attachment 385970 [details]
Mark test as failing in all platforms

Clearing flags on attachment: 385970

Committed r253684: <https://trac.webkit.org/changeset/253684>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/58043267>

[Chris Dumez]

(In reply to WebKit Commit Bot from comment #5)
> Comment on attachment 385970 [details]
> Mark test as failing in all platforms
> 
> Clearing flags on attachment: 385970
> 
> Committed r253684: <https://trac.webkit.org/changeset/253684>

Seems like we could have simply dumped its console messages to stderr instead of skipping it entirely, no?