Bug 222183 – [BigSur+ Wk2] imported/w3c/web-platform-tests/media-source/mediasource-seek-during-pending-seek.html is a flakey failure

• Component: New Bugs
• Status: RESOLVED WORKSFORME

[Amir Mark Jr]

imported/w3c/web-platform-tests/media-source/mediasource-seek-during-pending-seek.html

Is a flakey failure on BigSur Wk2

History:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmedia-source%2Fmediasource-seek-during-pending-seek.html

Text Diff:

--- /Volumes/Data/slave/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-seek-during-pending-seek-expected.txt
+++ /Volumes/Data/slave/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-seek-during-pending-seek-actual.txt
@@ -1,4 +1,6 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS Test seeking to a new location before transitioning beyond HAVE_METADATA.
-PASS Test seeking to a new location during a pending seek.
+TIMEOUT Test seeking to a new location during a pending seek. Test timed out

[Radar WebKit Bug Importer]

<rdar://problem/74528766>

[Amir Mark Jr]

Created attachment 421348 [details]
Patch

[Truitt Savell]

Comment on attachment 421348 [details]
Patch

Clearing flags on attachment: 421348

Committed r273335 (234487@main): <https://commits.webkit.org/234487@main>

[Jean-Yves Avenard [:jya]]

I haven't been able to reproduce the TIMEOUT error.

However, I can easily reproduce the error:
`
FAIL Test seeking to a new location before transitioning beyond HAVE_METADATA. assert_equals: Event types match. expected "seeked" but got "playing"
`

Upon review of the spec and how this test is written, the behaviour seen is correct. This test can intermittently fail and this is to be expected.

https://github.com/web-platform-tests/wpt/blob/89505713ad43568e8e9c985d489a59209aaebad6/media-source/mediasource-seek-during-pending-seek.html#L50

The test does the following.
mediaElement.play()
mediaElement.currentTime = non_buffered_area;
it will then append data at currenTime and make sure that the `seeked` event is fired before `playing`

The `playing` event is to be fired according to HTML5 spec when the internal play steps are running.
https://html.spec.whatwg.org/multipage/media.html#internal-play-steps

> 3. If the media element's paused attribute is true, then:
>     1. Change the value of paused to false.
>     2. If the show poster flag is true, set the element's show poster flag to false and run the time marches on steps.
>     3. Queue a media element task given the media element to fire an event named play at the element.
>     4. If the media element's readyState attribute has the value HAVE_NOTHING, HAVE_METADATA, or HAVE_CURRENT_DATA, queue a media element task given the media element to fire an event named waiting at the element.
>        Otherwise, the media element's readyState attribute has the value HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA: notify about playing for the element.
> 

The other non-normative reference to when the `playing` event is to be fired is when:
https://html.spec.whatwg.org/multipage/media.html#event-media-playing

> readyState is newly equal to or greater than HAVE_FUTURE_DATA and paused is false, or paused is newly false and readyState is equal to or greater than HAVE_FUTURE_DATA. Even if this event fires, the element might still not be potentially playing, e.g. if the element is paused for user interaction or paused for in-band content. 

mediaElement.play() was called early in the test, so mediaElement.paused attribute is false. So the next time the mediaElement should be firing the `playing` event is when readyState moves to a value equal or greater than HAVE_FUTURE_DATA.

Media Source Extension clarifies the following when it comes to seeking:
https://w3c.github.io/media-source/#mediasource-seeking

> 1. If new playback position is not in any TimeRanges of HTMLMediaElement.buffered
>         If the HTMLMediaElement.readyState attribute is greater than HAVE_METADATA, then set the HTMLMediaElement.readyState attribute to HAVE_METADATA.
>         Note
> 
>         Per HTMLMediaElement ready states [HTML] logic, HTMLMediaElement.readyState changes may trigger events on the HTMLMediaElement.
>         The media element waits until an appendBuffer() call causes the coded frame processing algorithm to set the HTMLMediaElement.readyState attribute to a value greater than HAVE_METADATA. 
> 


So per spec, when seeking to a non-buffered area, readyState will be set to HAVE_METADATA (which the test properly checks)
This is done while 
https://html.spec.whatwg.org/multipage/media.html#seeking :
> Wait until the user agent has established whether or not the media data for the new playback position is available, and, if it is, until it has decoded enough data to play back that position.

The HTML spec clearly specifies here that we are to wait for *decoded* data, not until data is available (e.g. buffered). 

Now the appendBuffer operation runs and the coded frame processing algorithm is run:
https://w3c.github.io/media-source/#dfn-coded-frame-processing

> 2. If the HTMLMediaElement.readyState attribute is HAVE_METADATA and the new coded frames cause HTMLMediaElement.buffered to have a TimeRanges for the current playback position, then set the HTMLMediaElement.readyState attribute to HAVE_CURRENT_DATA.
> Note
> 
> Per HTMLMediaElement ready states [HTML] logic, HTMLMediaElement.readyState changes may trigger events on the HTMLMediaElement.
> 
> 3. If the HTMLMediaElement.readyState attribute is HAVE_CURRENT_DATA and the new coded frames cause HTMLMediaElement.buffered to have a TimeRanges that includes the current playback position and some time beyond the current playback position, then set the HTMLMediaElement.readyState attribute to HAVE_FUTURE_DATA.
> 

So readyState during the appendBuffer and before `updateend` is fired should transition mediaElement.currentState from HAVE_METADATA to HAVE_FUTURE_DATA and fire the `playing` event.

So the order of operation when seeking in this test becomes:
1- wait until we have buffered data
2a- modify readyState to HAVE_FUTURE_DATA
2b- wait until we have enough decoded data
3- complete seeking.

When we complete the seeking operation, the `seeked` event will be fired.
Step 2a and 2b can be done in parallel, it is possible that we have enough decoded data at the same time the readyState is being modified.

so we can't assert that the `seeked` event will be fired before the `playing` event. The opposite should be a more regular occurrence.


I have opened a WPT bug:
https://github.com/web-platform-tests/wpt/issues/28984

[Robert Jenner]

Expanding expectations to included BigSur+:
https://trac.webkit.org/changeset/280208/webkit