Bug 229611

Summary: MediaRecorder with canvas captureStream() produces blank video
Product: WebKit Reporter: Ashley Gullen <ashley>
Component: WebRTCAssignee: Nobody <webkit-unassigned>
Severity: Normal CC: alex, amatsak, dahalls, eric.carlson, jerry, jonlee, kkinnunen, Kongpheng.Nanthavongsa, pauldconlin, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Safari Technology Preview   
Hardware: Unspecified   
OS: Unspecified   

Description Ashley Gullen 2021-08-27 08:34:44 PDT
Using MediaRecorder to record a MediaStream from the canvas captureStream() method appears to only produce a blank video in Safari 15 / TP 130 on macOS 12 beta (21A5304g).

Demo URL: https://downloads.scirra.com/labs/screenrecording/

Steps to reproduce:

1. Click 'Record canvas'
2. Wait a few seconds
3. Click 'Stop recording'

A video file will then be downloaded.

Observed result:
The video has the expected duration, but just displays all black.

Expected result:
Video to show canvas content that was recorded.

Other notes:
This appears to work correctly on iOS 14, so may be unique to macOS. Also note we have reports that a flickery video is recorded on Safari 14 on macOS, but we can't easily test that right now. Finally the 'Record screen' option - using getDisplayMedia() - appears to work correctly, so this looks like an issue specific to using MediaRecorder with canvas captureStream().
Comment 1 Radar WebKit Bug Importer 2021-09-03 08:35:18 PDT
Comment 2 Alexander Khovansky 2021-09-05 06:10:55 PDT
We ran into the same problem (except with a bit more erratic behavior) when using a canvas for realtime video modification.
It looks like it's not MediaRecorder specific but rather captureStream producing a bad stream.

Here's a codepen that demonstrates the issue:

It has a video source that is rendered to a canvas, which is then rendered to a different video element using captureStream. Works in Safari 14 but not in TP.
Comment 3 David Halls 2021-09-07 13:11:31 PDT
I'm also seeing this. It's captureStream generating blank video.

One example is https://experiments.withgoogle.com/webcam-displacement

This works on Safari 14.1.2 but not on TP 131 (both Big Sur).
Comment 4 youenn fablet 2021-09-13 02:34:20 PDT
> It looks like it's not MediaRecorder specific but rather captureStream
> producing a bad stream.

Right, this is also my understanding.
Comment 5 David Halls 2021-09-25 22:49:08 PDT
Broken on Safari 15 now :(
Comment 6 jerry 2021-10-18 14:29:30 PDT
How do we get the devs to at least confirm this?

This happens to the latest TP 133.
Comment 7 Kongpheng 2021-11-19 18:40:45 PST
Issue still reproducible on Safari version 15.1 (15612., 15612).

Another way to reproduce:
* Open https://webrtc.github.io/samples/src/content/capture/canvas-video/
* Using mouse, move teapot on the left side around, by holding down left-click and dragging your mouse around
* Notice that teapot on the right doesn't mirror the left teapot's movement

I wonder if issue is due to auto-play constraint violation.  Console logs follows.

MediaElementSession::clientWillBeginAutoplaying(74E6F415461C6466) state = Autoplaying
HTMLMediaElement::setShouldDelayLoadEvent(74E6F415461C6466) true
MediaElementSession::removeBehaviorRestriction(74E6F415461C6466) removed RequirePageConsentToLoadMedia
MediaElementSession::autoplayPermitted(74E6F415461C6466) Returning FALSE because element is not visible in the viewport
MediaElementSession::beginInterruption(74E6F415461C6466) state = Autoplaying, interruption type = InvisibleAutoplay, interruption count = 0
Comment 8 pauldconlin 2022-01-03 11:22:07 PST
Adding on here that this is affecting capture for Flipgrid in Safari 15. We can fall back to a standard video element -> MediaRecorder capture, but that removes our ability to provide all of the canvas enabled effects. For a quick fix for this bug, we are just disabling capture on Safari 15.

@youenn - anything else I can help provide to diagnose?
Comment 9 Jon Lee 2022-01-03 14:49:49 PST

*** This bug has been marked as a duplicate of bug 230613 ***