[MSE] SourceBuffer::provideMediaData() may remove sample from decode queue unexpectedly

[Daniel Zhang]

I found in some cases, a sample already in trackBuffer.decodeQueue could be removed unexpectedly when there is gap ahead of it. For example, if samples of the following ranges are already in the trackBuffer.decodeQueue,

    [0 .. 5.0) [10.0, ... 30.0)

when SourceBuffer::provideMediaData() is called on such a decodeQueue, it may remove sample 10.0 (decode time) due to one second gap. Then, if the gap is filled (segment re-appended) later, it will enqueue samples from this decodeQueue but without sample 10.0.

The problem usually happens after playback for a while, e.g. buffered [10.0, ... 30.0), then seeking to the beginning. During re-appending the first segments, provideMediaData() may remove sample 10.0. As the removed sample is usually a sync sample, it results in playback issue if the enqueued samples are sent to decoder (not flushed).

I suggest to move the following removal code to be after the one-second gap checking:

    trackBuffer.decodeQueue.erase(trackBuffer.decodeQueue.begin());

How do you think? Thanks.

Best Regards,
Daniel ZHANG

[Alicia Boya García]

I think you are right.

The sample should only be removed from the decodeQueue after it's certain that it will be enqueued to the decoders. (Or if a the entire decodeQueue has to be flushed because of a seek).

[Alicia Boya García]

Created attachment 350964 [details]
Patch

[Alicia Boya García]

Better late than never, here is a patch including a test case.

[Jer Noble]

Comment on attachment 350964 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=350964&action=review

> LayoutTests/media/media-source/media-source-append-acb-no-frame-lost.html:42
> +        let updateEnd = waitFor(sourceBuffer, 'updateend');
> +        run('sourceBuffer.appendBuffer(initSegment)');
> +        await updateEnd;

Just a little nit: because events will never be fired before the current run-loop yields, it's safe to restructure this to:

run('sourceBuffer.appendBuffer(initSegment)');
await waitFor(sourceBuffer, 'updateend');

(I wish we could go back in time and change appendBuffer(...) to return a Promise<void>.)

[Alicia Boya García]

Created attachment 350986 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 350986 [details]
Patch

Clearing flags on attachment: 350986

Committed r236566: <https://trac.webkit.org/changeset/236566>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/44840989>