Bug 230621

Summary: MediaStream canvas.captureStream() fails for Canvas Context2D
Product: WebKit Reporter: Kimmo Kinnunen <kkinnunen>
Component: CanvasAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: ansarman, btham, dino, mukeshbsws, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Local Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=230613
Attachments:
Description Flags
Frozen video stream repro video none

Description Kimmo Kinnunen 2021-09-22 06:18:06 PDT
MediaStream canvas.captureStream() fails for Canvas Context2D

see the bug 230617 test case mediastream-canvas-to-video.html last few elements
Comment 1 Kimmo Kinnunen 2021-09-22 06:18:49 PDT
Sorry, bug 230613.
Comment 2 Radar WebKit Bug Importer 2021-09-29 06:19:16 PDT
<rdar://problem/83668394>
Comment 3 Kimmo Kinnunen 2021-10-28 00:04:59 PDT
*** Bug 231598 has been marked as a duplicate of this bug. ***
Comment 4 Kimmo Kinnunen 2021-10-28 00:05:50 PDT
>This only occurs on Safari 15 and Safari Technology Preview. It does not occur on Safari 14.

> Here is a JSFiddle that reproduces the issue: https://jsfiddle.net/x8pz4u2y/1/. Click on the "Start" button to start capturing the canvas, which flashes between blue and green every second. Note that you may need to resize the window to get the video element to show in Safari 15 for some reason (which may be a separate bug but my main concern is that the resulting video is red).
Comment 5 btham 2021-12-01 10:23:55 PST
Hi Kimmo, I was wondering if there were any updates on this? (I am the original filer for https://bugs.webkit.org/show_bug.cgi?id=231598.)
Comment 6 Anastasiya Sarmant 2022-01-14 08:21:31 PST
Created attachment 449174 [details]
Frozen video stream repro video
Comment 7 Anastasiya Sarmant 2022-01-14 08:26:16 PST
Any updates on that bug?
We have developed webgl2 based background replacement feature for MS Teams and this bug prevents us from shipping it on Safari.
We have Macbook Air M1 and Macbook Pro Intel based. The result of the below code snippet is a bit different on those machines. On M1 the stream is frozen but I'm able to get frames flowing by randomly clicking the page or scrolling, it looks the canvas streams expects repaint or layout update. On Macbook Pro it's frozen regardless. Attached a video example.

// simple repro
public async processStream(mediaStream: MediaStream): Promise<MediaStream> { 
        const settings = mediaStream.getVideoTracks()[0].getSettings(); 
        const canvas = document.createElement('canvas'); 
        const ctx = canvas.getContext('2d'); 
        canvas.width = settings.width; 
        canvas.height = settings.height; 
        const video = document.createElement('video'); 
        video.srcObject = mediaStream; 
        video.autoplay = true; 
        video.onplay = (): void => { 
            const loop = (): void => { 
                if (!video.paused && !video.ended) { 
                    ctx.drawImage(video, 0, 0); 
                    setTimeout(loop,1000 /66);  
                } 
            }; 
            loop(); 
        }; 
        video.play(); 
        return canvas.captureStream(); 
    }
Comment 8 Mukesh 2022-01-14 10:35:30 PST
canvas.captureStream()is broken on Safari 15 and Safari Technology Preview. Any update on this or workaround to get a stream from canvas?