[GStreamer][EME] Encrypted, fragmented MP4s not supported

• Component: WebKitGTK
• Status: NEW ---

[Charlie Turner]

Placeholder bug to record a number of issues affecting encrypted, fragmented MP4s.

There are at least two WPT's that show this problem,

  clearkey-mp4-playback-temporary-encrypted-clear.https.html
  clearkey-mp4-playback-temporary-clear-encrypted.https.html

And their *-drm equivalents. These are files that contain an encrypted track, followed by a clear one and vice versa respectively.

The current GStreamer behaviour is to try and reconfigure the src pad caps from qtdemux to transition from application/x-cenc -> video/x-h264 and vice versa respectively.
This reconfiguration is not supported by decodebin2, it can't dynamically change it's pipeline topology at runtime like that. Decodebin3 should support that, but it at least has bugs dealing with this situation, and we're not ready to use it in MSE at the moment anyway, so that's a non-starter.

An attempt was made to modify qtdemux to treat a new fragmented track whose protection status differs from the preceding track as deserving of a new src pad. That works to the extent that decodebin2 will create a second DecodeChain, and autoplug an appropriate pipeline for the new situation, however WK MSE does not allow demuxers to do this for the same SourceBuffer:

connectDemuxerSrcPadToAppsinkFromStreamingThread: Only one stream per SourceBuffer is allowed! Ignoring stream 2 by adding a black hole probe

So we have some options:
  1) Wait until we can turn decodebin3 on always, even in MSE and fix any bugs there blocking this behaviour from working.
  2) Modify qtdemux to expose a new pad (stalled in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/merge_requests/180) and then modify MSE to allow multiple streams per SourceBuffer.
  3) Create a new bin handling video/x-h264 (and friends) *as well as* application/x-cenc with a rank higher than decodebin2. This new bin would essentially wrap decodebin2, and if necessary shove a decryptor in front of the decodebin2 when necessary (and remove it when necessary as fragments change protection status). The large downside here is you'd now have a bin-within-a-bin for every non-encrypted video. Proxypads introduce non-neglible performance hit, and it's just a pretty gross hack when the above two options would result in a cleaner codebase.


These sorts of files don't *appear* to be that popular in the wild, so I'm personally happy to wait for either 1 or 2. 3 is an option if an emergency case comes up where this needs implementing.