Bug 272521 – [MSE] Decoding errors when appending segment with B-frames and previous segment ends with an I-frame

• Status: RESOLVED
• Resolution: FIXED
• Component: Media

[Carlos Bentzen]

When a segment containing B-frames is appended (i.e. with out-of-display-order frames, and samples PTS and DTS values don't always match), WebKit has logic to erase frames from the previous segment and avoid decoding glitches (see bug 206412). However, the current logic falls short in one edge case: if the previous segment ends with an I-frame in the overlap region to be erased.

If the demuxer honors ISOBMFF edit lists, then the first I-frame from an incoming segment with B-frames can be placed earlier in decoding order than the last (I-)frame from the previous segment (potentially with different resolution), and that last I-frame doesn't get erased, and will be pushed for decoding. This confuses the decoder and the following P/B frames will fail to decode or decode incorrectly with artifacts due to missing/incorrect reference frame.

When this edge case is triggered:
- In Apple WebKit's, playback directly stops and we get an error event.
- In upstream GStreamer-based ports we don't have this problem due to ignoring edit lists (see bug 231019). But in downstream WPE ports where it's enabled, we get macroblock artifacts, but could potentially get fatal errors as well depending on the decoder plugin that's used.

Hosted test page to reproduce it manually: https://people.igalia.com/cadubentzen/webkit/out-of-order (EXPECTED: playback failure on Safari at 4s. For GStreamer-based ports, not reproducible upstream due to ignoring edit lists). It works with Firefox and Chromium.

A detailed analysis was done from a case that triggered this in https://github.com/WebPlatformForEmbedded/WPEWebKit/issues/1301 and https://github.com/WebPlatformForEmbedded/WPEWebKit/pull/1184. Credits to Vivek Arumugam.

I have patch with a possible fix and a layout test that reproduces this with `video/mock`, just need a bit more checking before opening for review. The patch is still only a mitigation as this scenario isn't well defined in the spec (https://github.com/w3c/media-source/issues/187).

[Carlos Bentzen]

Pull request: https://github.com/WebKit/WebKit/pull/27033

[EWS]

Committed 277532@main (ec49bb21041c): <https://commits.webkit.org/277532@main>

Reviewed commits have been landed. Closing PR #27033 and removing active labels.

[Radar WebKit Bug Importer]

<rdar://problem/126533350>