Bug 177663

Summary: LayoutTest media/W3C/video/events/event_progress_manual.html is a flaky failure
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: Tools / TestsAssignee: Peng Liu <peng.liu6>
Status: RESOLVED DUPLICATE    
Severity: Normal CC: ap, commit-queue, eric.carlson, jer.noble, jlewis3, lforschler, peng.liu6, sroberts
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=196929

Description Ryan Haddad 2017-09-29 09:42:17 PDT
LayoutTest media/W3C/video/events/event_progress_manual.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/r222647%20(4820)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2FW3C%2Fvideo%2Fevents%2Fevent_progress_manual.html

--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/media/W3C/video/events/event_progress_manual-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/media/W3C/video/events/event_progress_manual-actual.txt
@@ -6,6 +6,9 @@
 PASS true is true
 
 TEST COMPLETE
+PASS true is true
+
+TEST COMPLETE
 spec reference
Comment 1 Alexey Proskuryakov 2017-10-03 21:00:54 PDT
It is quite weird that this test prints completion lines twice. It must manage to dispatch the progress event before HTML finishes loading, which is why the first notifyDone() call doesn't work.

Also, super weird that we have a non-local URL here (http://media.w3.org/2010/05/video/movie_300?...). Loading it is blocked in DRT and WKTR, so I'm not sure if the test is even testing what it's supposed to.

Jer, Eric, what's the best way to avoid flakiness here? Perhaps we don't need this test at all, and can delete it?
Comment 2 Matt Lewis 2018-02-08 15:42:08 PST
This is still happening:

https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r228293%20(2835)/results.html
https://build.webkit.org/builders/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/builds/2835

The Dif is still the same. The platforms that the flak happens on is the macOS platform.
Comment 3 Matt Lewis 2018-02-08 16:00:51 PST
Marked as flaky in https://trac.webkit.org/changeset/228300/webkit/
Comment 4 Shawn Roberts 2019-04-16 11:07:28 PDT
Test has been failing on WK1 builds as well. 

Mojave WK1 Release just started showing in the last few days, but it's reproducible with older revisions that High Sierra and Mojave WK1 Debug.

It is causing failures on EWS. https://bugs.webkit.org/show_bug.cgi?id=196929

Same diff as before.
Comment 5 Shawn Roberts 2019-04-16 11:11:55 PDT
Updated test expectations for all of Mac in https://trac.webkit.org/changeset/244344/webkit
Comment 6 Shawn Roberts 2019-04-16 11:12:49 PDT
*** Bug 196929 has been marked as a duplicate of this bug. ***
Comment 7 Peng Liu 2019-10-10 14:21:09 PDT
This test case will fail when the video element fires more than one "progress" event before the test case complete.

*** This bug has been marked as a duplicate of bug 181966 ***