Bug 76280 - media/W3C/video/networkState/networkState_during_progress.html is flaky
Summary: media/W3C/video/networkState/networkState_during_progress.html is flaky
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Charlie Turner
URL:
Keywords: InRadar
: 82976 133865 200354 (view as bug list)
Depends on: 204070
Blocks:
  Show dependency treegraph
 
Reported: 2012-01-13 09:11 PST by James Robinson
Modified: 2019-11-11 08:20 PST (History)
18 users (show)

See Also:


Attachments
Patch (6.82 KB, patch)
2019-10-22 10:43 PDT, Charlie Turner
no flags Details | Formatted Diff | Diff
Patch for landing (6.82 KB, patch)
2019-10-22 13:38 PDT, Charlie Turner
no flags Details | Formatted Diff | Diff
Patch for landing (6.81 KB, patch)
2019-10-22 13:40 PDT, Charlie Turner
no flags Details | Formatted Diff | Diff
Patch for landing (9.76 KB, patch)
2019-10-22 14:05 PDT, Charlie Turner
no flags Details | Formatted Diff | Diff
Patch for landing (6.81 KB, patch)
2019-10-22 14:07 PDT, Charlie Turner
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Comment 1 Eric Carlson 2012-01-13 11:07:43 PST
One problem is that the expected results contain a failure:


videoElement.networkState should be NETWORK_LOADING during progress event

On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".


FAIL "1" should be 2. Was 1.

TEST COMPLETE
spec reference
Comment 2 Eric Carlson 2012-01-13 11:13:16 PST
I *think* this test is wrong. It fails if video.networkState is not NETWORK_LOADING (2) when a progress event fires, but progress events are fired asynchronously. The "failure" (in the expected results) is that networkState is NETWORK_IDLE (1), but that just means that loading has completed.
Comment 3 Eric Carlson 2012-01-13 11:21:41 PST
Hmm, the final 'progress' event is supposed to be fired synchronously:

↪ Once the entire media resource has been fetched (but potentially before any of it has been decoded)
       Fire a simple event named progress at the media element.

 So this may well be a bug in WebKit.
Comment 4 James Robinson 2012-01-13 12:28:02 PST
(In reply to comment #1)
> One problem is that the expected results contain a failure:
> 
> 
> videoElement.networkState should be NETWORK_LOADING during progress event
> 
> On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
> 
> 
> FAIL "1" should be 2. Was 1.
> 
> TEST COMPLETE
> spec reference

Sometimes we get the PASS output, sometimes we get the (currently expected) failure message.
Comment 5 Balazs Kelemen 2013-05-20 01:39:42 PDT
This is also flaky with the GStreamer backend (at least with my soon to be uploaded patch), i.e. sometimes we got the correct output. Is it ok if I mark it flakey in the general TestExpectations file?
Comment 6 Balazs Kelemen 2013-05-23 03:34:29 PDT
Marked flakey in http://trac.webkit.org/changeset/150579
Comment 7 Russell Epstein 2019-08-06 08:54:06 PDT
*** Bug 200354 has been marked as a duplicate of this bug. ***
Comment 8 Alexey Proskuryakov 2019-08-06 09:23:02 PDT
rdar://problem/53825672
Comment 9 Alexey Proskuryakov 2019-08-06 09:26:18 PDT
*** Bug 82976 has been marked as a duplicate of this bug. ***
Comment 10 Charlie Turner 2019-10-22 10:43:01 PDT
Created attachment 381561 [details]
Patch
Comment 11 Charlie Turner 2019-10-22 10:45:15 PDT
The reason for the flake was because we were scheduling the last progress event on an async event queue, and then setting networkState to IDLE causing a race on JS receipt of the event. This is the case on all platforms so I have adjusted the expectations accordingly.
Comment 12 Eric Carlson 2019-10-22 10:51:27 PDT
Comment on attachment 381561 [details]
Patch

Thanks for the fix!
Comment 13 Charlie Turner 2019-10-22 13:38:40 PDT
Created attachment 381591 [details]
Patch for landing
Comment 14 Charlie Turner 2019-10-22 13:40:15 PDT
Created attachment 381592 [details]
Patch for landing
Comment 15 Charlie Turner 2019-10-22 13:40:39 PDT
*** Bug 133865 has been marked as a duplicate of this bug. ***
Comment 16 Charlie Turner 2019-10-22 14:05:16 PDT
Created attachment 381595 [details]
Patch for landing
Comment 17 Charlie Turner 2019-10-22 14:07:33 PDT
Created attachment 381597 [details]
Patch for landing
Comment 18 WebKit Commit Bot 2019-10-22 14:19:47 PDT
The commit-queue encountered the following flaky tests while processing attachment 381592 [details]:

inspector/runtime/getProperties.html bug 203271 (authors: bburg@apple.com, drousso@apple.com, and joepeck@webkit.org)
The commit-queue is continuing to process your patch.
Comment 19 WebKit Commit Bot 2019-10-22 14:49:30 PDT
Comment on attachment 381597 [details]
Patch for landing

Clearing flags on attachment: 381597

Committed r251460: <https://trac.webkit.org/changeset/251460>
Comment 20 WebKit Commit Bot 2019-10-22 14:49:32 PDT
All reviewed patches have been landed.  Closing bug.
Comment 21 Charlie Turner 2019-10-24 16:28:23 PDT
This is still flaking on Mac debug builds,

https://build.webkit.org/results/Apple%20Mojave%20Debug%20WK1%20%28Tests%29/r251541%20%286472%29/media/W3C/video/networkState/networkState_during_progress-pretty-diff.html

Sometimes I see an incorrect network state as well, but the test is clearly bugged since it's expected that it exits after one event. Perhaps we should just remove it? I noted that neither Chrome nor Firefox are spec-compliant in a similar test case either.

I can't cross check on GTK atm, because the debug bots exit early due to the many crashes there, and I don't have a powerful enough machine to build that for a few days.
Comment 22 Jer Noble 2019-11-11 08:20:45 PST
This caused a crash on macOS and iOS platforms. We need a way to de-flake this test without doing synchronous messaging when handling a message from the platform.