Bug 230623 – MediaStream canvas.captureStream() playback to LocalSampleBufferDisplayLayer is flaky

• Component: Media
• Status: RESOLVED FIXED

[Kimmo Kinnunen]

MediaStream canvas.captureStream() playback to LocalSampleBufferDisplayLayer is flaky

Sometimes the system is not creating the AVSampleBufferDisplayLayer because the samples arrive 
before the media player determines whether accelerated rendering is possible.

The callbacks that update HTMLMediaElement::m_cachedSupportsAcceleratedRendering are run asynchronously and too late.

[Kimmo Kinnunen]

To observe,  bug 230613 should be fixed and load:

https://webrtc.github.io/samples/src/content/capture/canvas-video/

[Radar WebKit Bug Importer]

<rdar://problem/83668578>

[Kimmo Kinnunen]

*** Bug 235176 has been marked as a duplicate of this bug. ***

[Kimmo Kinnunen]

Created attachment 449646 [details]
testcase

[Kimmo Kinnunen]

To work around this problem, one might try to assign or re-assign the stream in a different JavaScript callback than the one that creates the video element.

[youenn fablet]

Created attachment 449649 [details]
Patch

[youenn fablet]

Created attachment 449663 [details]
Patch

[Kimmo Kinnunen]

Comment on attachment 449663 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=449663&action=review

It is a step forward for the users, which is good.
The reenqueue logic is broken in many ways and not very good pattern for correct software and the class has numerous race conditions. We should spend some more time writing tests and fixing those for this class, at some point.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:284
> +    Locker locker { AdoptLock, m_currentVideoSampleLock };

This is like saying  "Try to send the sample. if it's a bit inconvenient, maybe you can just skip sending it."
Either the sample is needed for correct operation and send it or it's not needed and it should not be sent.
This  especially  does not make sense in this "reenqueue" phase. 
So you can consider using unconditional locking.

[youenn fablet]

Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:284
> > +    Locker locker { AdoptLock, m_currentVideoSampleLock };
> 
> This is like saying  "Try to send the sample. if it's a bit inconvenient,
> maybe you can just skip sending it."

Not really, the idea is more: if it is locked, a new sample is coming, so do not bother enqueuing a sample, since a fresh one will be enqueued very shortly.
It is true that, sometimes, we will reenqueue several times the same sample.

Also we need to lock as m_currentVideoSample is guarded by m_currentVideoSampleLock.

[EWS]

Committed r288435 (246324@main): <https://commits.webkit.org/246324@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 449663 [details].