Bug 176929

Summary: Layout Test imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks.html is flaky
Product: WebKit Reporter: Matt Lewis <jlewis3>
Component: MediaAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: cdumez, commit-queue, eric.carlson, ryanhaddad, tsavell, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=201588
Attachments:
Description Flags
Mark test as failing in all platforms none

Description Matt Lewis 2017-09-14 11:24:22 PDT 
imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks.html is flaky on El Capitan WK2 according to the flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Fready-states%2Fautoplay-with-slow-text-tracks.html

current failure:
https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/r222025%20(4461)/results.html
https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/4461

diff:

--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks-actual.txt
@@ -1,4 +1,3 @@
-CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
 CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
 CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
 CONSOLE MESSAGE: line 31: TypeError: null is not an object (evaluating 'track.track.cues.length')
Comment 2 Truitt Savell 2018-10-05 11:38:26 PDT 
This test has been flakey on Mojave. It seems to not be a flakey failure on many of the other platforms anymore. The diff is the same as it was previously. test is extremely flaky.
Comment 3 Ryan Haddad 2018-10-23 10:23:05 PDT 
Marked test as flaky in https://trac.webkit.org/r237353
Comment 4 youenn fablet 2019-12-18 07:06:04 PST 
Created attachment 385970 [details]
Mark test as failing in all platforms
Comment 5 WebKit Commit Bot 2019-12-18 08:08:34 PST 
Comment on attachment 385970 [details]
Mark test as failing in all platforms

Clearing flags on attachment: 385970

Committed r253684: <https://trac.webkit.org/changeset/253684>
Comment 6 WebKit Commit Bot 2019-12-18 08:08:35 PST 
All reviewed patches have been landed.  Closing bug.
Comment 7 Radar WebKit Bug Importer 2019-12-18 08:09:26 PST 
<rdar://problem/58043267>
Comment 8 Chris Dumez 2019-12-18 08:25:29 PST 
(In reply to WebKit Commit Bot from comment #5)
> Comment on attachment 385970 [details]
> Mark test as failing in all platforms
> 
> Clearing flags on attachment: 385970
> 
> Committed r253684: <https://trac.webkit.org/changeset/253684>

Seems like we could have simply dumped its console messages to stderr instead of skipping it entirely, no?