Bug 76280

Summary:

media/W3C/video/networkState/networkState_during_progress.html is flaky

Product:

WebKit

Reporter:

James Robinson <jamesr>

Component:

Tools / Tests

Assignee:

Charlie Turner <cturner>

Status:

REOPENED ---

Severity:

Normal

CC:

ap, calvaris, cdumez, commit-queue, cturner, enrica, eocanha, eric.carlson, esprehn+autocc, ews-watchlist, glenn, gyuyoung.kim, jer.noble, kbalazs, philipj, repstein, sergio, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

528+ (Nightly build)

Hardware:

Unspecified

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=200354

Bug Depends on:

204070

Bug Blocks:

Attachments:

Description	Flags
Patch	none
Patch for landing	none
Patch for landing	none
Patch for landing	none
Patch for landing	none

Description James Robinson 2012-01-13 09:11:52 PST

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=media%2FW3C%2Fvideo%2FnetworkState%2FnetworkState_during_progress.html&showExpectations=true

Comment 1 Eric Carlson 2012-01-13 11:07:43 PST

One problem is that the expected results contain a failure:


videoElement.networkState should be NETWORK_LOADING during progress event

On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".


FAIL "1" should be 2. Was 1.

TEST COMPLETE
spec reference

Comment 2 Eric Carlson 2012-01-13 11:13:16 PST

I *think* this test is wrong. It fails if video.networkState is not NETWORK_LOADING (2) when a progress event fires, but progress events are fired asynchronously. The "failure" (in the expected results) is that networkState is NETWORK_IDLE (1), but that just means that loading has completed.

Comment 3 Eric Carlson 2012-01-13 11:21:41 PST

Hmm, the final 'progress' event is supposed to be fired synchronously:

↪ Once the entire media resource has been fetched (but potentially before any of it has been decoded)
       Fire a simple event named progress at the media element.

 So this may well be a bug in WebKit.

Comment 4 James Robinson 2012-01-13 12:28:02 PST

(In reply to comment #1)
> One problem is that the expected results contain a failure:
> 
> 
> videoElement.networkState should be NETWORK_LOADING during progress event
> 
> On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
> 
> 
> FAIL "1" should be 2. Was 1.
> 
> TEST COMPLETE
> spec reference

Sometimes we get the PASS output, sometimes we get the (currently expected) failure message.

Comment 5 Balazs Kelemen 2013-05-20 01:39:42 PDT

This is also flaky with the GStreamer backend (at least with my soon to be uploaded patch), i.e. sometimes we got the correct output. Is it ok if I mark it flakey in the general TestExpectations file?

Comment 6 Balazs Kelemen 2013-05-23 03:34:29 PDT

Marked flakey in http://trac.webkit.org/changeset/150579

Comment 7 Russell Epstein 2019-08-06 08:54:06 PDT

*** Bug 200354 has been marked as a duplicate of this bug. ***

Comment 8 Alexey Proskuryakov 2019-08-06 09:23:02 PDT

rdar://problem/53825672

Comment 9 Alexey Proskuryakov 2019-08-06 09:26:18 PDT

*** Bug 82976 has been marked as a duplicate of this bug. ***

Comment 10 Charlie Turner 2019-10-22 10:43:01 PDT

Created attachment 381561 [details]
Patch

Comment 11 Charlie Turner 2019-10-22 10:45:15 PDT

The reason for the flake was because we were scheduling the last progress event on an async event queue, and then setting networkState to IDLE causing a race on JS receipt of the event. This is the case on all platforms so I have adjusted the expectations accordingly.

Comment 12 Eric Carlson 2019-10-22 10:51:27 PDT

Comment on attachment 381561 [details]
Patch

Thanks for the fix!

Comment 13 Charlie Turner 2019-10-22 13:38:40 PDT

Created attachment 381591 [details]
Patch for landing

Comment 14 Charlie Turner 2019-10-22 13:40:15 PDT

Created attachment 381592 [details]
Patch for landing

Comment 15 Charlie Turner 2019-10-22 13:40:39 PDT

*** Bug 133865 has been marked as a duplicate of this bug. ***

Comment 16 Charlie Turner 2019-10-22 14:05:16 PDT

Created attachment 381595 [details]
Patch for landing

Comment 17 Charlie Turner 2019-10-22 14:07:33 PDT

Created attachment 381597 [details]
Patch for landing

Comment 18 WebKit Commit Bot 2019-10-22 14:19:47 PDT

The commit-queue encountered the following flaky tests while processing attachment 381592 [details]:

inspector/runtime/getProperties.html bug 203271 (authors: bburg@apple.com, drousso@apple.com, and joepeck@webkit.org)
The commit-queue is continuing to process your patch.

Comment 19 WebKit Commit Bot 2019-10-22 14:49:30 PDT

Comment on attachment 381597 [details]
Patch for landing

Clearing flags on attachment: 381597

Committed r251460: <https://trac.webkit.org/changeset/251460>

Comment 20 WebKit Commit Bot 2019-10-22 14:49:32 PDT

All reviewed patches have been landed.  Closing bug.

Comment 21 Charlie Turner 2019-10-24 16:28:23 PDT

This is still flaking on Mac debug builds,

https://build.webkit.org/results/Apple%20Mojave%20Debug%20WK1%20%28Tests%29/r251541%20%286472%29/media/W3C/video/networkState/networkState_during_progress-pretty-diff.html

Sometimes I see an incorrect network state as well, but the test is clearly bugged since it's expected that it exits after one event. Perhaps we should just remove it? I noted that neither Chrome nor Firefox are spec-compliant in a similar test case either.

I can't cross check on GTK atm, because the debug bots exit early due to the many crashes there, and I don't have a powerful enough machine to build that for a few days.

Comment 22 Jer Noble 2019-11-11 08:20:45 PST

This caused a crash on macOS and iOS platforms. We need a way to de-flake this test without doing synchronous messaging when handling a message from the platform.