Bug 154189

Summary: Adaptive Stream Video Throws DOM Exception 18 When Uploaded to a Texture
Product: WebKit Reporter: jonobr1 <jonobrandel>
Component: CanvasAssignee: Nobody <webkit-unassigned>
Status: RESOLVED DUPLICATE    
Severity: Critical CC: dino, electroteque, jer.noble, jonlee, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Safari 9   
Hardware: Mac   
OS: OS X 10.11   
Attachments:
Description Flags
Screenshot of Chrome (left) and Safari (right) behavior. none

Description jonobr1 2016-02-12 13:04:45 PST
Created attachment 271208 [details]
Screenshot of Chrome (left) and Safari (right) behavior.

It's really common practice to use a <video /> element to play some content. Less frequent, but still very useful is to take that elements content and put it in a canvas. In this example I'm using the <video /> element, but pointing it to a stream via the library Shaka Player (https://github.com/google/shaka-player). I expect to be able to take the pixel information of the <video /> element at a given time and reproduce it / manipulate it on a canvas element. However, in Safari 9 this is not the case. When implementing with WebGL I get a DOM Exception 18 error. When implementing with canvas I get no error, but also no image. The assets are same domain, so I don't think there are CORS issues present. Below are links that reproduce this behavior:

1) Go to either:

Example using canvas via texImage2D in WebGL (with Three.js):
https://dl.dropboxusercontent.com/u/1595444/shaka-player/stream-texture.html

Example using canvas via drawImage directly:
https://dl.dropboxusercontent.com/u/1595444/shaka-player/shaka-canvas.html

2) Click the "Play Button" on the <video /> element.
3) The video starts playing

When going through these steps in Google Chrome when I hit play I see 2 videos on the page. One that is the <video /> element and one that is the <canvas /> element that reproduces the pixel information of the source <video /> element. I expect Safari to behave similarly.
Comment 1 Radar WebKit Bug Importer 2016-02-12 14:19:11 PST
<rdar://problem/24636250>
Comment 2 Jer Noble 2016-02-12 14:21:38 PST
This looks to be the same bug as #135379.  Because loading is done by AVFoundation and not WebKit, we don't have an opportunity to verify CORS headers, and thus all cross-domain resources will taint canvas and WebGL contents when painting.

*** This bug has been marked as a duplicate of bug 135379 ***
Comment 3 jonobr1 2016-02-12 14:40:21 PST
Thanks for the quick reply! I'm still trying to understand the issue in the other bug, but from a preliminary look I think that bug is different. In the case of this issue the error is being thrown despite the contents being on the same domain. In addition, the <canvas /> element demo (https://dl.dropboxusercontent.com/u/1595444/shaka-player/shaka-canvas.html) throws no errors but does not transfer content.

I'm not an engineer and just a simple web developer so my understanding of how DOM -> rendering is very naive, but to me the issue seems to be in the irregularity of the <video /> elements behavior.
Comment 4 jonobr1 2016-02-12 14:52:07 PST
Also, I believe this was previously working before I updated to Safari 9. I need to confirm this though. Are there archived versions of Safari available?
Comment 5 Jer Noble 2016-02-12 15:26:08 PST
Aha, this is a slightly different, but related, issue than the one in #135379. The CORS check fails because the scheme used for the <video> element ("blob://") does not match the one used for the page ("http://").  Additionally, nothing is painted to canvas because our MSE back-end does not currently support painting. We'll have to re-write the way decoding works to get access to the decoded frames before they're added to a AVSampleBufferDisplayLayer, in order to be able to paint efficiently. I raised this issue with Richard in the linked bug.
Comment 6 Jer Noble 2016-02-12 15:26:42 PST
And no, I suspect that this never worked in any Safari.
Comment 7 Jer Noble 2016-02-12 15:36:42 PST
Looks like the correct dup for this bug is #125157.

*** This bug has been marked as a duplicate of bug 125157 ***
Comment 8 jonobr1 2016-02-12 16:53:12 PST
Ahh, this is awesome. Thanks so much for the fast reply and information!
Comment 9 Daniel Rossi 2016-02-13 14:49:20 PST
This is a matter of leaving things to the last minute. An accident waiting to happen. 

It is related to the CORS flaw. Once CORS support is enabled , MediaSource WebGL textures will still be a problem. A very basic usage. 

See this for reference

https://bugs.webkit.org/show_bug.cgi?id=135379

My proxy hack work around does not work on Dash only mp4 files.