[MediaStream] Limit the number of remote video samples queued

• Component: Media
• Status: ASSIGNED ---

[Eric Carlson]

MediaPlayerPrivateMediaStreamAVFObjC::removeOldSamplesFromPendingQueue releases queued video samples that are earlier than the current stream time. This works for samples from a local media stream track, but frames from a remote track are tagged for "immediate display" and don't have valid time stamps so there is no such thing as an "old" sample.

[Eric Carlson]

<rdar://problem/33223015>

[Eric Carlson]

Created attachment 315435 [details]
Proposed patch.

[youenn fablet]

Comment on attachment 315435 [details]
Proposed patch.

View in context: https://bugs.webkit.org/attachment.cgi?id=315435&action=review

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:263
> +    if (!queue.first()->decodeTime().isValid()) {

I applied the patch locally to check whether  decodeTime is invalid in the case of RealtimeIncomingVideoSource.
It is valid but is negative. At least on my Mac, we are not going through that loop.
We will probably go through the next while loop and empty the queue there.
Is it different on iOS?

[Eric Carlson]

(In reply to youenn fablet from comment #3)
> Comment on attachment 315435 [details]
> Proposed patch.
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=315435&action=review
> 
> > Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:263
> > +    if (!queue.first()->decodeTime().isValid()) {
> 
> I applied the patch locally to check whether  decodeTime is invalid in the
> case of RealtimeIncomingVideoSource.
> It is valid but is negative. At least on my Mac, we are not going through
> that loop.
> We will probably go through the next while loop and empty the queue there.
> Is it different on iOS?

While the raw PTS time is negative in the CMSampleBuffer:

(lldb) po (CMSampleBuffer)0x00007f88b20ca290
CMSampleBuffer 0x7f88b20ca290 retainCount: 1 allocator: 0x7fff8383df40
  ...
	numSamples = 1
	sampleTimingArray[1] = {
		{PTS = {-1/1 = -1.000}, DTS = {-1/1 = -1.000}, duration = {INVALID}},
	}
	sampleAttachmentsArray[0] = {
		sample 0:
			DisplayImmediately = true
	}
	imageBuffer = 0x7f88b5c2f140

"decodeTime" is invalid:

(lldb) p mediaSample.decodeTime()
(WTF::MediaTime) $11 = { Invalid } {
   = (m_timeValue = 0, m_timeValueAsDouble = 0)
  m_timeScale = 0
  m_timeFlags = '\0'
}
(lldb) p mediaSample.decodeTime().isValid()
(bool) $12 = false

But we should probably also check explicitly for negative times as well.

[Eric Carlson]

Created attachment 315457 [details]
Patch for landing.

[WebKit Commit Bot]

Comment on attachment 315457 [details]
Patch for landing.

Clearing flags on attachment: 315457

Committed r219513: <http://trac.webkit.org/changeset/219513>