RESOLVED FIXED 217578
Webcam video from navigator.mediaDevices.getUserMedia() to 2D canvas fails on Safari on iPhone 
https://bugs.webkit.org/show_bug.cgi?id=217578
Summary Webcam video from navigator.mediaDevices.getUserMedia() to 2D canvas fails on...
jujjyl
Reported 2020-10-11 04:47:28 PDT
STR: <html><head></head><body><canvas></canvas><script> navigator.mediaDevices.getUserMedia({ audio: false, video: true }).then((stream) => { var video = document.createElement('video'); video.srcObject = stream; video.play(); var canvas = document.querySelector('canvas'); var c2d = canvas.getContext('2d'); function draw() { canvas.width = video.videoWidth; canvas.height = video.videoHeight; c2d.drawImage(video, 0, 0, video.videoWidth, video.videoHeight); requestAnimationFrame(draw); } requestAnimationFrame(draw); }); </script></body></html> Works on desktop Firefox, Chrome and Edge, and on mobile Android Firefox and Chrome, but produces a black canvas on Safari.
Attachments
Patch (6.46 KB, patch)
2020-10-13 03:56 PDT, youenn fablet 		no flags
DetailsFormatted DiffDiff
Patch for landing (6.49 KB, patch)
2020-10-13 09:12 PDT, youenn fablet 		no flags
DetailsFormatted DiffDiff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2020-10-11 10:50:40 PDT
<rdar://problem/70183875>
youenn fablet
Comment 2 2020-10-12 03:51:15 PDT
I reproed the issue. This does not repro if the video element is rendering.
youenn fablet
Comment 3 2020-10-13 00:40:10 PDT
Here is a fiddle that reproduces the issue: https://jsfiddle.net/kw8fbcma/4/ Capture probably fails as the samples are queued in the local sample buffer display layer which is not actually rendering.
youenn fablet
Comment 4 2020-10-13 03:56:29 PDT
Created attachment 411209 [details] Patch
Eric Carlson
Comment 5 2020-10-13 08:11:52 PDT
Comment on attachment 411209 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=411209&action=review > Source/WebCore/platform/graphics/MediaPlayer.cpp:968 > +void MediaPlayer::setVisibleForCanvas(bool b) > +{ > + m_visible = b; > + m_private->setVisibleForCanvas(b); s/b/visible/ > Source/WebCore/platform/graphics/MediaPlayerPrivate.h:99 > + virtual void setVisibleForCanvas(bool b) { setVisible(b); } Ditto
youenn fablet
Comment 6 2020-10-13 09:12:52 PDT
Created attachment 411223 [details] Patch for landing
EWS
Comment 7 2020-10-13 10:06:21 PDT
Committed r268398: <https://trac.webkit.org/changeset/268398> All reviewed patches have been landed. Closing bug and clearing flags on attachment 411223 [details].
Simon Fraser (smfr)
Comment 8 2020-10-13 11:30:28 PDT
Why no layout test?
youenn fablet
Comment 9 2020-10-13 11:47:57 PDT
(In reply to Simon Fraser (smfr) from comment #8) > Why no layout test? It would be nice but I found it hard to test. Ideas welcome. We would basically need to verify that AVSampleBufferDisplayLayer is not keeping references too much video samples in its queue but I do not see APIs to get that info. Ideally, we would want to validate that the mock device CVPixelBufferPool is able to recycle its pixel buffers at some reasonable point in time. That would make the mock very close to the real camera. Not sure how we can do that though. Maybe we can observe the CVPixelBuffer pointers.
Geoffrey Garen
Comment 10 2020-10-15 12:23:10 PDT
What happens if you do this: setVisibleForCanvas(true); setVisible(true); ? I think you do not become visible, because of the early return in MediaPlayerPrivateMediaStreamAVFObjC::setVisible. And maybe there are other early returns like this?
youenn fablet
Comment 11 2020-10-15 12:39:50 PDT
(In reply to Geoffrey Garen from comment #10) > What happens if you do this: > setVisibleForCanvas(true); > setVisible(true); > ? > > I think you do not become visible, because of the early return in > MediaPlayerPrivateMediaStreamAVFObjC::setVisible. And maybe there are other > early returns like this? For MediaPlayerPrivateMediaStreamAVFObjC, the MediaPlayer::setVisibleForCanvas(true) call is a no-op: - MediaPlayerPrivateMediaStreamAVFObjC::setVisibleForCanvas is a no-op. - MediaPlayer::m_visible is not used by MediaPlayerPrivateMediaStreamAVFObjC. The MediaPlayer::setVisible(true) call would end up setting MediaPlayerPrivateMediaStreamAVFObjC::m_visible to true. For other MediaPlayerPrivateXX, this patch is hopefully a no-op as MediaPlayer::setVisibleForCanvas is the same as MediaPlayer::setVisible. I'll double check this tomorrow but I think we can remove MediaPlayer::m_visible.
youenn fablet
Comment 12 2020-10-16 01:38:16 PDT
> I'll double check this tomorrow but I think we can remove > MediaPlayer::m_visible. Filed https://bugs.webkit.org/show_bug.cgi?id=217810
Note You need to log in before you can comment on or make changes to this bug.