Bug 192629 – [MediaStream] A stream's first video frame should be rendered

• Component: WebRTC
• Status: RESOLVED FIXED

[Eric Carlson]

When a MediaStream is attached to a video element, the first video frame should be rendered as soon as it is available even if the element is not playing.

Discussion at https://github.com/w3c/mediacapture-main/issues/555#issuecomment-446560648.

[Radar WebKit Bug Importer]

<rdar://problem/46664353>

[Eric Carlson]

Created attachment 357355 [details]
Patch

[youenn fablet]

Comment on attachment 357355 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=357355&action=review

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:283
> +    int m_waitingForFirstImage { 0 };

I would keep m_shouldDisplayFirstVideoFrame as a bool.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:487
> +    m_backgroundLayer.get().hidden = hidden;

one liner.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:1003
> +                m_waitingForFirstImage = 6;

Since the issue is with AVVideoCaptureSource, I would for now throw away if needed at AVVideoCaptureSource level.
This would not penalize other sources like canvas capture or remote tracks.

> LayoutTests/fast/mediastream/media-stream-renders-first-frame.html:36
> +                }, 500);

Do we need this setTimeout?

[EWS Watchlist]

Comment on attachment 357355 [details]
Patch

Attachment 357355 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/10405396

New failing tests:
imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-srcObject.https.html
webrtc/video-addTransceiver.html
webrtc/video.html
fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html
webrtc/video-addTrack.html
fast/mediastream/resize-letterbox.html
webrtc/video-interruption.html
fast/mediastream/MediaStream-video-element-displays-buffer.html
http/tests/media/media-stream/getusermedia-with-canvas.html
webrtc/video-addLegacyTransceiver.html

[EWS Watchlist]

Created attachment 357366 [details]
Archive of layout-test-results from ews107 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 357355 [details]
Patch

Attachment 357355 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/10405502

New failing tests:
imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-srcObject.https.html
webrtc/video-with-receiver.html
webrtc/video-remote-mute.html
webrtc/video-addTransceiver.html
webrtc/video.html
webrtc/video-addTrack.html
webrtc/video-interruption.html
webrtc/video-addLegacyTransceiver.html
webrtc/video-with-data-channel.html

[EWS Watchlist]

Created attachment 357373 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[Eric Carlson]

Created attachment 357475 [details]
Patch for landing

[EWS Watchlist]

Comment on attachment 357475 [details]
Patch for landing

Attachment 357475 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/10445762

New failing tests:
http/tests/media/media-stream/getusermedia-with-canvas.html

[EWS Watchlist]

Created attachment 357484 [details]
Archive of layout-test-results from ews104 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6

[Eric Carlson]

Created attachment 357509 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 357509 [details]
Patch for landing

Clearing flags on attachment: 357509

Committed r239319: <https://trac.webkit.org/changeset/239319>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.