Bug 160685 - Video tags can not be used for input to Canvas/WebGL on iOS
Summary: Video tags can not be used for input to Canvas/WebGL on iOS
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: iPhone / iPad iOS 9.3
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-09 02:07 PDT by Gregg Tavares
Modified: 2016-09-01 15:04 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gregg Tavares 2016-08-09 02:07:01 PDT
At some point it seems like Safari on iOS made it so video playback always happens fullscreen.

This is a problem for apps that are trying to use the video a source to a canvas or a webgl texture.

I don't know what problem was trying to be solved here. Maybe it was trying to make sure videos have a nice UX. Unfortunately it seems to have broken apps that use video as an input source like p5.js does often.

Is there any chance this can be fixed? Maybe check if the video is `display: none` and then let it play in the background? Or some new attribute that tells iOS Safari the video will be used for processing and not for direct on screen playback?

Note: people have found one hacky workaround which is as follows

make both a video and audio tag. Set both to the same source. Play the audio tag. Every rAF do `video.currentTime = audio.currentTime`. This magically works but seems very fragile and hacky.

Example http://webglsamples.org/video/video.html
Comment 1 randolphzeng 2016-08-31 13:48:30 PDT
I am using webgl to create a simple pre-viewing for my groups. We are doing scientific visualization and wants to have an idea how our films project to a dome will look like. This bug really affects what I wanna accomplish.
Comment 2 Jon Lee 2016-08-31 15:12:18 PDT
The previous policy allowed inline playback on iPads. Only on iPhones did video playback always go fullscreen. This policy has existed for a very long time, so when you say "it seems to have broken apps" are you saying something very recently broke?

For Safari 10 you can specify that a video should play inline by adding the "playsinline" attribute.
Comment 3 randolphzeng 2016-09-01 15:04:21 PDT
(In reply to comment #2)
> The previous policy allowed inline playback on iPads. Only on iPhones did
> video playback always go fullscreen. This policy has existed for a very long
> time, so when you say "it seems to have broken apps" are you saying
> something very recently broke?
> 
> For Safari 10 you can specify that a video should play inline by adding the
> "playsinline" attribute.

Yes, I am developing that recently. The hacky thing I am doing right now for iphone is that I have to manually set the video.currentTime(It actually doesn't need to have audio tag though). However, I wish I can have a more robust way to do this in previous mobile safari. Is there any way of doing that??
Thanks!