Bug 181663 - Video Element cannot playback local Canvas.captureStream on iOS
Summary: Video Element cannot playback local Canvas.captureStream on iOS
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Safari 11
Hardware: Unspecified iOS 11
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2018-01-15 19:03 PST by Adam
Modified: 2021-06-28 15:22 PDT (History)
14 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adam 2018-01-15 19:03:29 PST
If you use Canvas.captureStream and pass the stream to a video element srcObject you just get a blank video. It works in Safari on Mac OS, just not on iOS.

Example code: 

  const videoEl = document.createElement('video');

  const canvas = document.createElement('canvas');
  const ctx = canvas.getContext('2d');
  canvas.width = 640;
  canvas.height = 480;

  const randomColour = () => {
    return Math.round(Math.random() * 255);
  };

  setInterval(function() {
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    ctx.fillStyle = `rgb(${randomColour()}, ${randomColour()}, ${randomColour()})`;
    ctx.fillRect(0, 0, canvas.width, canvas.height);
  }, 1000);

  document.body.appendChild(canvas);
  document.body.appendChild(videoEl);

  videoEl.srcObject = canvas.captureStream(1);
  videoEl.setAttribute('playsinline', '');
  videoEl.play();

You can see the example running at: http://jsbin.com/nopusa

You are expected to see a Canvas element and a Video Element both with random colours being updated every 1 second. But on iOS it updates on the Canvas but not on the Video Element.
Comment 1 Brion Vibber 2019-05-26 14:59:00 PDT
I have the same problem under iOS 12.3; tested iPad Pro 9.7" and iPhone X.

It sees the stream and sets the <video> dimensions correctly, but never shows anything but solid black.
Comment 2 Brion Vibber 2019-05-26 15:04:03 PDT
On a whim I tested in Safari on the iPhone Simulator and Adam's sample works there. I suspect the sample buffer pixel layout used by the capture isn't playable by native iOS's media server, whereas the simulator piggybacks on the macOS one for actual video decoding.

I've seen problems like that when manually decoding video to uncompressed video sample buffers on iOS, where not all formats that work on the simulator work on the native devices...

In this case, it may be fixable by doing a pixel format conversion on the frame (or capturing it directly to a format that works).
Comment 3 Radar WebKit Bug Importer 2019-05-26 22:16:15 PDT
<rdar://problem/51150406>
Comment 4 Joe 2020-04-18 21:22:12 PDT
I can confirm that this is still an issue in iOS 13.3.1.

Here is another example that works with Safari on Mac but not iOS:

https://webrtc.github.io/samples/src/content/capture/canvas-video/
Comment 5 Kahiro Okina 2020-09-11 01:02:46 PDT
I can confirm that this is still an issue in iOS 14 beta8

Here is another example that works with Safari on Mac but not iOS

https://codepen.io/k-okina/pen/QWNrWwM
Comment 6 cabada 2020-09-18 08:45:19 PDT
I can confirm that this is still an issue in iOS 14 public release.
Comment 7 Davy De Durpel 2020-10-21 08:02:13 PDT
I can confirm that this is still an issue in iOS 14.1 and iOS 14.2 Public beta
Comment 8 Sam Wray 2021-01-18 07:31:49 PST
Confirmed still an issue in iOS 14.3.

Anything blocking this from being worked on?
We've just surpassed three years of this API being available and not working in Safari on iOS.
Comment 9 Francesco 2021-04-19 06:27:20 PDT
still an issue on 14.4.2

Any plan to take this in consideration?
For us it's necessary for the "virtual background" feature that is much appreciated during this pandemic period.
Comment 10 David Halls 2021-06-05 00:15:45 PDT
Bug is still present. It'd be better if captureStream() just threw an exception (or didn't exist) if iOS isn't going to support it.
Comment 11 Brion Vibber 2021-06-10 12:31:54 PDT
(In reply to David Halls from comment #10)
> Bug is still present. It'd be better if captureStream() just threw an
> exception (or didn't exist) if iOS isn't going to support it.

Works in Safari in iPadOS 15 beta on 11” iPad Pro now. Haven’t tested on any iPhones yet.
Comment 12 Brion Vibber 2021-06-10 16:20:13 PDT
Seems to work on iPhone X as well on iOS 15 beta now. Yay!