Bug 222115

Summary: <video> rendered inside <template> shows black for webcam stream (Safari 14.0.3 only)
Product: WebKit Reporter: Alex <asashayko>
Component: WebRTCAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: calvaris, cdumez, changseok, eric.carlson, esprehn+autocc, ews-watchlist, glenn, gyuyoung.kim, jer.noble, philipj, rniwa, ryansolid, sergio, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Safari 14   
Hardware: Mac (Intel)   
OS: macOS 11   
See Also: https://bugs.webkit.org/show_bug.cgi?id=271629
Attachments:
Description Flags
Patch
none
Patch none

Description Alex 2021-02-18 09:12:01 PST
Hello there!

Important notice - this is happening only on Safari 14.0.3.

After macOS and Safari update (14.0 -> 14.0.3) video stream from the webcam I'm getting using navigator.mediaDevices.getUserMedia shows only black inside <video> for some time and updates to the actual video at some point (sometimes not). This is situated inside <template> which I add to DOM using JavaScript.

In short:
<template><video></video></template> (shows black)
<video></video> (shows ok)

JsFiddle with the reproducible demo - https://jsfiddle.net/alex_oliynyk/402ed6wq/54/

Gif with the issue - https://share.getcloudapp.com/2NuElvYl

Also, I can confirm that video is actually playing fine. This is part of the app where I send frames from the video to the backend for processing and I get a successful response. This means that frames are not just black rectangulars but are proper images.

Best regards,
Comment 1 Radar WebKit Bug Importer 2021-02-19 18:42:56 PST
<rdar://problem/74546659>
Comment 2 Ryan 2021-05-14 20:46:21 PDT
Reproduced this black screen on M1 Mac as well.
https://codesandbox.io/s/streaming-user-device-dodse
Video seems to come in when the window loses and regains focus.

Template cloning also seems to also affect autoplay behavior of any video.
https://codesandbox.io/s/broken-autoplay-safari-vrr40?file=/src/index.js

I noticed this since it breaks in JavaScript frameworks that rely on this method for cloning nodes to do their templating. Lit uses `document.importNode` which seems to be fine, a number of other frameworks use this method. I'm the maintainer of Solid(https://github.com/solidui/solid) one such framework affected by this.
Comment 3 youenn fablet 2021-05-17 02:01:23 PDT
I cannot reproduce on my iMacPro with STP.
As of back screen on M1 Mac, this is handled internally.
@Ryan, can you try the latest BigSur 11.3 update to see whether the issue is fixed for you on the M1?
Comment 4 Ryan 2021-05-18 01:02:35 PDT
Just updated to 11.3.1 and Safari 14.1 (16611.1.21.161.6). I can still reproduce in all the posted reproductions.
Comment 5 youenn fablet 2021-05-18 01:09:48 PDT
Thanks Ryan, I split your issues as separate issues as they might be M1 specific.
@Alex, are you still seeing this issue. Is it M1 specific?
Comment 6 Ryosuke Niwa 2021-05-18 02:23:07 PDT
I think this is basically https://bugs.webkit.org/show_bug.cgi?id=222657.
Comment 7 youenn fablet 2021-05-19 07:22:18 PDT
I had a look and it seems that the media element thinks it is hidden, hence why rendering does not work.
This probably explains also the autoplay restriction.
Comment 8 Ryan 2021-05-19 20:31:35 PDT
I also reproduced this in Safari 14.1 on my Intel Macbook Pro on Catalina (work computer).
Comment 9 youenn fablet 2021-05-20 00:57:59 PDT
Created attachment 429145 [details]
Patch
Comment 10 Ryosuke Niwa 2021-05-20 02:48:39 PDT
Comment on attachment 429145 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429145&action=review

> LayoutTests/fast/media/template-hidden.html:19
> +    myVideo.srcObject = await navigator.mediaDevices.getUserMedia({video:true});
> +    await myVideo.play(); 

Do we really need to use getUserMedia here? I think a regular video will be enough?
Comment 11 youenn fablet 2021-05-20 02:51:58 PDT
(In reply to Ryosuke Niwa from comment #10)
> Comment on attachment 429145 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=429145&action=review
> 
> > LayoutTests/fast/media/template-hidden.html:19
> > +    myVideo.srcObject = await navigator.mediaDevices.getUserMedia({video:true});
> > +    await myVideo.play(); 
> 
> Do we really need to use getUserMedia here? I think a regular video will be
> enough?

The test does not need it as it uses internals API.

When running the test manually, it is safer to use this code path since the renderer is different from regular video streaming.

I'll update the patch to have it run in WK1 as well by making the getUserMedia call optional.
Comment 12 youenn fablet 2021-05-20 03:14:40 PDT
Created attachment 429153 [details]
Patch
Comment 13 Alex 2021-05-20 06:17:12 PDT
(In reply to youenn fablet from comment #5)
> Thanks Ryan, I split your issues as separate issues as they might be M1
> specific.
> @Alex, are you still seeing this issue. Is it M1 specific?

Hi!

Yes, still happening for me at Safari 14.1 (16611.1.21.161.6) macOS Big Sur 11.3.1, Intel-based Macbook Pro.

Cheers!
Comment 14 Eric Carlson 2021-05-20 07:44:21 PDT
Comment on attachment 429153 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429153&action=review

> LayoutTests/fast/media/template-hidden.html:17
> +    if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia)

Isn’t this test reversed?
Comment 15 youenn fablet 2021-05-20 08:28:51 PDT
Comment on attachment 429153 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429153&action=review

>> LayoutTests/fast/media/template-hidden.html:17
>> +    if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia)
> 
> Isn’t this test reversed?

Should be fine as is, we call getUserMedia if defined and return undefined otherwise
Comment 16 EWS 2021-05-20 08:33:19 PDT
Committed r277783 (237944@main): <https://commits.webkit.org/237944@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 429153 [details].