Bug 192629

Summary: [MediaStream] A stream's first video frame should be rendered
Product: WebKit Reporter: Eric Carlson <eric.carlson>
Component: WebRTCAssignee: Eric Carlson <eric.carlson>
Status: RESOLVED FIXED    
Severity: Normal CC: commit-queue, ews-watchlist, jer.noble, rniwa, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Local Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews107 for mac-sierra-wk2
none
Archive of layout-test-results from ews126 for ios-simulator-wk2
none
Patch for landing
none
Archive of layout-test-results from ews104 for mac-sierra-wk2
none
Patch for landing none

Description Eric Carlson 2018-12-12 09:38:03 PST
When a MediaStream is attached to a video element, the first video frame should be rendered as soon as it is available even if the element is not playing.

Discussion at https://github.com/w3c/mediacapture-main/issues/555#issuecomment-446560648.
Comment 1 Radar WebKit Bug Importer 2018-12-12 09:38:23 PST
<rdar://problem/46664353>
Comment 2 Eric Carlson 2018-12-14 16:41:46 PST
Created attachment 357355 [details]
Patch
Comment 3 youenn fablet 2018-12-14 17:03:42 PST
Comment on attachment 357355 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=357355&action=review

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:283
> +    int m_waitingForFirstImage { 0 };

I would keep m_shouldDisplayFirstVideoFrame as a bool.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:487
> +    m_backgroundLayer.get().hidden = hidden;

one liner.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:1003
> +                m_waitingForFirstImage = 6;

Since the issue is with AVVideoCaptureSource, I would for now throw away if needed at AVVideoCaptureSource level.
This would not penalize other sources like canvas capture or remote tracks.

> LayoutTests/fast/mediastream/media-stream-renders-first-frame.html:36
> +                }, 500);

Do we need this setTimeout?
Comment 4 EWS Watchlist 2018-12-14 18:02:36 PST
Comment on attachment 357355 [details]
Patch

Attachment 357355 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/10405396

New failing tests:
imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-srcObject.https.html
webrtc/video-addTransceiver.html
webrtc/video.html
fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html
webrtc/video-addTrack.html
fast/mediastream/resize-letterbox.html
webrtc/video-interruption.html
fast/mediastream/MediaStream-video-element-displays-buffer.html
http/tests/media/media-stream/getusermedia-with-canvas.html
webrtc/video-addLegacyTransceiver.html
Comment 5 EWS Watchlist 2018-12-14 18:02:38 PST
Created attachment 357366 [details]
Archive of layout-test-results from ews107 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6
Comment 6 EWS Watchlist 2018-12-14 18:36:31 PST
Comment on attachment 357355 [details]
Patch

Attachment 357355 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/10405502

New failing tests:
imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-srcObject.https.html
webrtc/video-with-receiver.html
webrtc/video-remote-mute.html
webrtc/video-addTransceiver.html
webrtc/video.html
webrtc/video-addTrack.html
webrtc/video-interruption.html
webrtc/video-addLegacyTransceiver.html
webrtc/video-with-data-channel.html
Comment 7 EWS Watchlist 2018-12-14 18:36:32 PST
Created attachment 357373 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 8 Eric Carlson 2018-12-17 14:14:17 PST
Created attachment 357475 [details]
Patch for landing
Comment 9 EWS Watchlist 2018-12-17 15:14:57 PST
Comment on attachment 357475 [details]
Patch for landing

Attachment 357475 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/10445762

New failing tests:
http/tests/media/media-stream/getusermedia-with-canvas.html
Comment 10 EWS Watchlist 2018-12-17 15:14:58 PST
Created attachment 357484 [details]
Archive of layout-test-results from ews104 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6
Comment 11 Eric Carlson 2018-12-17 17:50:05 PST
Created attachment 357509 [details]
Patch for landing
Comment 12 WebKit Commit Bot 2018-12-17 21:31:30 PST
Comment on attachment 357509 [details]
Patch for landing

Clearing flags on attachment: 357509

Committed r239319: <https://trac.webkit.org/changeset/239319>
Comment 13 WebKit Commit Bot 2018-12-17 21:31:32 PST
All reviewed patches have been landed.  Closing bug.