iOS 8 / OSX 10.10 WebGL - using a video from an other domain fails (CORS bug)

• Component: Media
• Status: RESOLVED FIXED

[Klaus Reinfeld]

When trying to use a video as WebGL texture and that video is located on an other domain, then there will be wrong 'Security Error'.

The CORS / crossOrigin settings are all set correctly, the bug is inside Webkit.

See here a simple example for testing:
http://krpano.com/ios/bugs/ios8-webgl-video-cors/

This fails currently on iOS 8 beta iPads and also in the current OSX 10.10 Desktop Safari. The current Android Chrome 36 has the same bug (just as note if internally some code has been shared). All other Browsers are fine.

[Alex Christensen]

This would be fixed by implementing MediaPlayerPrivateAVFoundationObjC::didPassCORSAccessCheck.  See https://bugs.webkit.org/show_bug.cgi?id=99037

[Klaus Reinfeld]

iOS 8 release and still not fixed!
Very bad!

[Radar WebKit Bug Importer]

<rdar://problem/18370945>

[Dean Jackson]

This is a media bug. Our AVFoundation backend does not check CORS for tainting.

[Radar WebKit Bug Importer]

<rdar://problem/18375542>

[Klaus Reinfeld]

Still not fixed in iOS 8.0.2!

[Klaus Reinfeld]

iOS 8.1 beta and still not fixed...

It seems to be hopeless...
Is it really that hard to fix that bug or what's the problem with it? Why doesn't it get fixed? No developer resources?

[Alexey Proskuryakov]

There is no reason to post status of every iOS beta or update - there is no fix here in this bug yet. Nothing is going to change before this bug is resolved.

[Yuri Zapuchlak]

We are experiencing the same issue and would love to have the web standard 'crossorigin' attribute be supported on the Video element in IOS and Mac Safari.

I wish I would have found this bug report before I created my own test page (https://vidmaker.com/test/safari_crossorigin_test) to reproduce the issue, hehe.

The workaround we've had to employ at the moment to support our HTML5 video editor (https://vidmaker.com) in Safari is to proxy all of our videos stored in AWS S3 through nginx running on one of our servers just for Safari users. It works for the moment, but it will not at any sort of scale (unless we continue to add more proxy servers).

[Sean]

I'd like to chime in that this is a problem for many 360 video players including our own. Would be great to get a fix!

[Richard Leider]

This issue is a blocker for YouTube 360 and 3D videos in webkit.

[Klaus Reinfeld]

This bug is now almost one year old and there were more than ten iOS updates since then without any fixes.

It will be interesting to see if something will change now when a big player like Google is requesting a fix...

[Klaus Reinfeld]

Still not fixed in iOS 9 beta 1...

[Richard Leider]

YouTube 360 videos ( https://www.youtube.com/watch?v=edcJ_JNeyhg ) recently shipped for the versions of Firefox and Opera that support MSE.  It would be great to bring 3D and 360 videos to WebKit browsers that support MSE, too.

[Richard Leider]

YouTube 360 videos (https://www.youtube.com/watch?v=BNqW6uE-Q_o) now work on IE where IE supports Media Source Extensions.  It would be great to bring 3D and 360 videos to WebKit browsers that support MSE, too.

[Denny]

This also fails on most recent desktop Safari 9.0 (11601.1.56).

[Tong Shen]

Hey guys,

I am looking into this.

I have confirmed that implementing MediaPlayerPrivateAVFoundationObjC::didPassCORSAccessCheck by always returning true can make the example work, so what we need to do is properly implementing this function.

I'm studying https://bugs.webkit.org/show_bug.cgi?id=99037 and try to figure out how to do the same for MediaPlayerPrivateAVFoundationObjC::didPassCORSAccessCheck.

Any code pointers/suggestions is more than welcome!

Thanks!

[Tong Shen]

Created attachment 266976 [details]
Proof of concept patch

[Tong Shen]

Hey guys,

Put together a proof of concept patch for this. Verified on OSX 10.11, the demo works in MiniBrowser. Hooray!

Is there any reviewer willing to look at this? I know very little about Webkit (never hacked on Webkit before), so let's evaluate the approach first. If it's OK, then we move on to coding style/webkit details/reusing previous GStreamer code.

Thank you very much :D

=== TL;DR ===

The following is implementation detail of the proof of concept patch.

AVAssetResourceLoader only calls its delegate when the AVURLAsset does not start with http or https. Also, it does not handle CORS check.

So, the situation leaves us 3 possible solutions:

1. do not use AVFoundation APIs. Well that does not seems like the right way to go.

2. wait for AVFoundation to handle CORS check. I personally do not want to wait. Besides, AVFoundation API is for general purpose Internet media resource loading (not limited to web browsers), while CORS checking is a very specific use case in web browsers. For this reason, I think we should handle CORS on Webkit side instead of AVFounation side. We already have mature and well-tested code for CORS, right?

3. hack into AVFoundation API, perform the CORS check somewhere in the progress.

This patch uses approach 3. Implementation is a little hacky.

- We replace http/https URLs with mock protocols mock-http/mock-https, so AVAssetResourceLoader will call our delegate
- In the delegate, use CachedResourceLoader to get response, perform CORS check.
- If CORS check passes, fill response data into the original AVAssetResourceLoadingRequest.

Except for the first step, the latter 2 steps look exactly like what we did for GStreamer backend: http://trac.webkit.org/changeset/167073/ , which is another reason I am happy about it.

[Jer Noble]

Comment on attachment 266976 [details]
Proof of concept patch

View in context: https://bugs.webkit.org/attachment.cgi?id=266976&action=review

Thanks for the patch. We've previously looked into this mechanism for custom data loading of media resources, and our experiments revealed a number of problems with this approach.  Namely:

This will only work for initial requests.  Subsequent requests (for additional byte ranges, for sub-resources, or due to HTTP redirects) will not come through this path, and will thus we will not get a chance to do a CORS check on those requests.  We have to assume that these requests would not pass CORS, or we would risk opening up a hole in CORS support.  Additionally, this path is only triggered for the HLS manifest load, but requests for HLS media segments does not come through this API.

I am going to give this a r- due to the above, but only due to the shortcomings of the approach in the patch, not because of the patch itself. In addition, there are a few things in your patch I'd like to call out that would need to be fixed in the hypothetical case that the approach was valid.

> avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:412
> +    int m_didPassCORSAccessCheck;

This should be defined in terms of an enum, with defined states. Such as:

enum CORSAccessCheckResult {
    CORSAccessUnknown,
    CORSAccessDenied,
    CORSAccessAllowed,
};
CORSAccessCheckResult m_didPassCORSAccessCheck { CORSAccessUnknown };

> avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:514
> +    , m_didPassCORSAccessCheck(0)

With C++11, this style of initialization can be put in the header.

> avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:839
> +    if (parsedURL.protocol().lower() == "http") {
> +        parsedURL.setProtocol("mock-http");
> +    } else if (parsedURL.protocol().lower() == "https") {
> +        parsedURL.setProtocol("mock-https");
> +    }

WebKit coding style guidelines <https://webkit.org/code-style-guidelines/> specify dropping braces on single-line control statements.

> avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:2222
> +bool MediaPlayerPrivateAVFoundationObjC::didPassCORSAccessCheck() const
> +{
> +    //ASSERT(m_didPassCORSAccessCheck != 0);
> +    return m_didPassCORSAccessCheck == 2;

These should use the enum values i mentioned above.  So this would become:

return m_didPassCORSAccessCheck == CORSAccessAllowed;

> avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:2227
> +void MediaPlayerPrivateAVFoundationObjC::didFinishCORSAccessCheck(bool passed) {
> +    m_didPassCORSAccessCheck = passed ? 2 : 1;
> +}

And this would become:

m_didPassCORSAccessCheck = passed ? CORSAccessCheckAllowed : CORSAccessCheckDenied;

> avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:3445
> +    URL url([m_avAsset resolvedURL]);
> +    if (url.protocol() == "mock-http") {
> +        url.setProtocol("http");
> +    } else if (url.protocol() == "mock-https") {
> +        url.setProtocol("https");
> +    }
> +    return url;

Ditto about the WebKit coding style guidelines.

> avfoundation/objc/WebCoreAVFResourceLoader.mm:75
> +    if (requestURL.protocol() == "mock-http") {
> +        requestURL.setProtocol("http");
> +    } else if (requestURL.protocol() == "mock-https") {
> +        requestURL.setProtocol("https");
> +    }

Ditto.

> avfoundation/objc/WebCoreAVFResourceLoader.mm:83
> +    m_origin = loader->document() ? loader->document()->securityOrigin() : nullptr;
> +    ASSERT(m_origin.get());

No need to add .get() to a RefPtr or RetainPtr when testing nullity.  Those classes provide a convenience bool operator.

> avfoundation/objc/WebCoreAVFResourceLoader.mm:117
> +    bool didPassCORSAccessCheck = resource->passesAccessControlCheck(*m_origin.get());

Also, they override operator*(), so no need to add a .get() before dereferencing.

[Jer Noble]

(In reply to comment #15)
> YouTube 360 videos (https://www.youtube.com/watch?v=BNqW6uE-Q_o) now work on
> IE where IE supports Media Source Extensions.  It would be great to bring 3D
> and 360 videos to WebKit browsers that support MSE, too.

Richard, would YouTube 360 videos use MSE on Safari desktop?

[Richard Leider]

Yes, YouTube would use MSE for 360 videos on Safari desktop.

[Tong Shen]

(In reply to comment #20)

Hi Jer, thanks a lot for the reply! Clarify things a lot. A few questions though, still hoping we can find a way to fix this:

> This will only work for initial requests.  Subsequent requests (for
> additional byte ranges, for sub-resources, or due to HTTP redirects) will
> not come through this path, and will thus we will not get a chance to do a
> CORS check on those requests.  

Do you mean subsequent requests will not reach code in MediaPlayerPrivateAVFoundationObjC, or CachedResourceLoader does not give us a chance to handle subsequent requests?

Either way, if so, how did we handle this situation in other backends (namely GStreamer backend, as in https://bugs.webkit.org/show_bug.cgi?id=99037 )? Did we leave a loophole there?...
 
> Additionally, this path is only triggered for the HLS manifest load, but
> requests for HLS media segments does not come through this API.

Is this caused by AVFoundation API? If so, that would be a blocker.

Thanks again for your time!

[Jer Noble]

(In reply to comment #23)
> (In reply to comment #20)
> 
> Hi Jer, thanks a lot for the reply! Clarify things a lot. A few questions
> though, still hoping we can find a way to fix this:
> 
> > This will only work for initial requests.  Subsequent requests (for
> > additional byte ranges, for sub-resources, or due to HTTP redirects) will
> > not come through this path, and will thus we will not get a chance to do a
> > CORS check on those requests.  
> 
> Do you mean subsequent requests will not reach code in
> MediaPlayerPrivateAVFoundationObjC, or CachedResourceLoader does not give us
> a chance to handle subsequent requests?

Well, there's a related problem I haven't mentioned: if you simply do the CORS check, and notify AVFoundation of the final URL (provided CORS passes), then AVFoundation will proceed to use that final URL for all new requests, and thus you won't get a chance to handle subsequent requests due to redirects, additional sub-resources, etc.  If you provide the data directly, you fall into a separate performance category:

The AVAssetResourceLoaderDelegate API was designed for on-disk resources which were not stored as files which AVFoundation could parse directly. E.g., database entries, UUEncoded e-mail attachments, etc.  So AVFoundation makes a lot of decisions about buffering, enqueueing load requests, etc. based on the assumption that the file is entirely available and exists locally (on disk). So doing the "bait-and-switch" with "mock-http://" breaks a lot of features, such as when the "canplaythrough" event fires, how much pre-buffering occurs, and more.

Looking more carefully at the patch, I see that is exactly the trap this approach falls into.

> Either way, if so, how did we handle this situation in other backends
> (namely GStreamer backend, as in
> https://bugs.webkit.org/show_bug.cgi?id=99037 )? Did we leave a loophole
> there?...

No I think this is a problem specific to AVFoundation.

> > Additionally, this path is only triggered for the HLS manifest load, but
> > requests for HLS media segments does not come through this API.
> 
> Is this caused by AVFoundation API? If so, that would be a blocker.

Yes it is.

> Thanks again for your time!

No problem. Rest assured we care very much about making this work in Safari.

[Tong Shen]

(In reply to comment #24)

Aha, I rest my case:-) Seems this can only be fixed in AVFoundation side.

BTW the performance decision thing really teaches me a lesson about improving user experience. Kudos on that!

Thank you, and look forward to seeing this bug resolved (hopefully soon :D )

[Jer Noble]

(In reply to comment #22)
> Yes, YouTube would use MSE for 360 videos on Safari desktop.

In that case, there's the bigger issue: MSE-backed <video> elements cannot currently paint to Canvas or WebGL. That's a known issue which we are also working on.

[Richard Leider]

Is there an existing bug/feature request I could cc myself on that represents the known issue about MSE-backed <video> elements being unable to paint to Canvas or WebGL?

[Daniel Rossi]

Hi somebody mentioned about doing the proxy work around. This works fine for capturing image data uris from the canvas but it is not working for webGL. It still complains about a security issue with texImage2D

ie

"SecurityError: DOM Exception 18: An attempt was made to break through the security policy of the user agent."



You can test with this example file and changing your hosts to match the sub-domain exactly which is silly it should be matching top-level domains.

flowplayer.electroteque.org/video/bitrate/big_buck_bunny_600k.mp4

Any ideas if this crossorigin attr is finally getting in there it's years a bit too late.

[Daniel Rossi]

The proxy work around is totally no good for production and defeats the purpose of using CDN's for videos ! My nginx proxy path has to redirect to cloudfront internally. 

I suggest a 2D video alternative plays back, and then the user can turn on 360 video and play back the proxy video. If this even works of course! 

The fact that it has to match the sub domain exactly is the issue here. Any wonder why Youtube can't even support Safari yet ?

[Daniel Rossi]

Excuse me. Confirming the proxy work around works. I forgot not only sub domains need to match but also ports. 

I believe this may require what I suggested. A 2D version of the video with user controls to turn on 360 which will need to run through the proxy. 

So for Safari and IE and anything with no crossorigin attribute, play 2D, and turn on 360 on demand. Unfortunately there is no way to make a 360 video a flat plane without webgl on.

This is one of the most important features to have not implemented and it kind of cripples html5 video a lot ironically requiring FLASH !  The proxy is the only known work around. I needed this for a video capture feature when extracting the image data uri from the canvas. 

I believe the corssorigin attribute is required on TextTracks too while I'm at it.

[Daniel Rossi]

here is a working example with the silly fallback hacks. 

It will replace the video with a "2D" version of the video, then a button will switch to the proxy source. 

as far as UI goes it as good as it gets rather than attempt to use the proxy source by default. 

https://flowplayer.electroteque.org/vr360/fp6

I hope this serious non compliant bug fix manages to find its way into OSX and Safari soon. 

How could something as important as this manage to be evaded for so many years I wonder ? 

BTW this bug tracker server is blocking my ip for some reason all of a sudden. I have to use a VPN to get access !

[David]

Any updates on this bug?? I have confirmed the CORS rules are not respected in safari for the latest iOS 9. The only workaround we were able to use was to create a proxy to our CDN so the videos are loaded from the same domain. But that's pointless. We have also tried 301/302 redirects but to no avail.

Firefox, chrome and edge all support it now. Safari is lagging behind.

We really need a solution for this. Can someone advise on a workaround?

[Daniel Rossi]

This "bug" is a standards flaw. It has been a problem for years. 

This is a serious show stopper for WebVR. At a time Apple is trying to show off their new Iphone VR case, this makes it a complete farce. 

WebVR has been exploding very rapidly over the past year and Safari is dragging it's heels making it completely unusable. 

Something as trivial as this should have been added years ago.

[Daniel Rossi]

I've had to do the same proxy workaround for video snapshot features but a single frame on a reverse proxy is much different than streaming through it. It won't scale and makes a CDN pointless. It is not a production ready solution. 

The only solution is to add the crossOrigin property on the video tag and while you are at it on the text tracks also. 

https://flowplayer.electroteque.org/snapshot/fp6

[Daniel Rossi]

Inline video is just another issue altogether and for another day. WebVR is unusable full stop on Iphone because of it :)

[David]

Interestingly, I have images working fine with the crossOrigin set to anonymous, but video seems to be the culprit. 

Amazing that Apple just hired an entire VR team to explore this explosive industry, and their flagship phone and browser are generations behind the competition. C'mon guys. This case is still assigned to "Nobody". We've already done all the work for you. Chop chop!

[David]

FYI, even MS Edge now supports the video textures and CORS correctly for webgl with the latest update (MS Edge 10586 TH2 Update). So all browsers seem to correctly support it, except Safari which is lacking proper support. 

Can we get an update on this bug when it will be fixed for Safari on MacOS and iOS?

[Jer Noble]

(In reply to comment #37)
> Can we get an update on this bug when it will be fixed for Safari on MacOS
> and iOS?

Apple does not comment on the timing or content of future releases. If you would like to register your desire to have this bug fixed, and to be notified when a fix is available in a public build, please file a Radar, and include your use case which is blocked by the current behavior.

[Daniel Rossi]

Yeah VR on IOS is a complete farce right now. A complete show stopper. 

CORS has been on images for a while now. Video has been broken for years. 

Sheer arrogance from Apple.

[Jer Noble]

(In reply to comment #39)
> Yeah VR on IOS is a complete farce right now. A complete show stopper. 
> 
> CORS has been on images for a while now. Video has been broken for years. 
> 
> Sheer arrogance from Apple.

David and Daniel, this is not a blog comment section. If you would like to comment about Apple's arrogance and how this bug is proof of it, I recommend you do so on Twitter. As the Apple employee who is working on this bug, you can accuse me of arrogance there at @jernoble.

Otherwise, if you do have constructive additions to this bug, (e.g., test cases, patches, etc.) please continue to post them.

[Jer Noble]

*** Bug 154189 has been marked as a duplicate of this bug. ***

[Daniel Rossi]

I will take this up with Apple because this has gone on for too long. This should take priority.

This flaw has been an issue for years and years. 

Only that people now are only realising it, working with webgl. It was reported in July 2014 and nobody is even assigned to it that just shows contempt. 

It is only just going to keep getting duped now.  

I'm not treating this issue as a forum its serious. 

Again this should be an embarrassment for Apple because it completely disables VR. 

If there is a fix I would be happy to test it, I am closely monitoring this.

[Daniel Rossi]

https://bugs.webkit.org/show_bug.cgi?id=125157

I believe these two are related. WebGL and canvas video drawing does not work with MediaSource / mpeg dash even with the reverse proxy work around which normally works on mp4. I have tested this myself and runs into the CORS security error. 

These should be merged into one. They are the same problem. 

More and more people once they find out trying to do VR on Safari are going to start making duplicates. 

I can't understand why that ticket is assigned and this one which is far more serious and the actual problem is not.

[David]

Just letting developers know that after updating to iOS 9.2.1, I have confirmed that this bug is still not yet fixed.

On a good note, my iPad is running only slightly slower than expected. So that's good.

[David]

Can someone verify if this bug has been fixed in the new iOS 9.3 released today?

[Alexey Proskuryakov]

No need to check that.

The bug is in NEW state, meaning that it hasn't been fixed in WebKit trunk. That will change before it's fixed in shipping software.

[Klaus Reinfeld]

Just finished updating and testing  - unfortunately again nothing had been fixed, neither the CORS bug nor the bad video-to-WebGL-texture performance...

[Daniel Rossi]

It may take years to get into an IOS update unless there is mass duplicate tickets with Apple. I have made a ticket but they closed it pending a duplicate with a much earlier ticket number. Which means the bug ticket has been there for years. 

It seems Firefox on IOS also uses and relies on webkit which is why it also is broken with CORS on IOS. 

Even with a CORS fix guys you still have to deal with the inline video playback issue with Iphone. It will prevent viewing the webgl canvas. I have no idea how to get through to Apple's thick heads about that one but I tried. 

You must wait until webkit nightly has a fix then hassle Apple I guess.

[Daniel Rossi]

So once webkit is fixed kindly ask Firefox to update their webkit code. I have a ticket for this here already, no point wasting time. If Firefox works Safari can just go jump in the bin. 

https://bugzilla.mozilla.org/show_bug.cgi?id=1256083

[Daniel Rossi]

Please keep track of this ticket, it seems to be related. They are doing their best it seems to fix it even though its years too late. They missed the opportunity already and way behind because they have never bothered to fix CORS and this video inline playback issue. 

Apple should be interfering with this but don't give a damn because they are hopeless. 

https://bugs.webkit.org/show_bug.cgi?id=125157#c29

[Daniel Rossi]

Any ideas of the status of this ?  I sent this to the other ticket which has dropped off the radar for two months now. 

I know it's pretty much crept up and steamrolled, many people trying to do WebVR including Youtube and Facebook of all people are also waiting. 

They should pay to make this happen. 

This problem has been left exposed for many years now and was just an accident waiting to happen. For some reason a few of us have to do the run around for many others. 

Obviously the reverse proxy work around is not ok for production so not even Facebook or Youtube are doing that and therefore Safari is broken. 

If there is a nightly fix to test please let me know.

[Alexey Proskuryakov]

Please see <https://bugzilla.mozilla.org/page.cgi?id=etiquette.html>.

[Zeno Crivelli]

Ok, I haven't tested it yet, but this is apparently fixed in Safari 10 on macOS:

https://twitter.com/zenoc/status/742770789880111104

(but not yet on iOS)

[Klaus Reinfeld]

Yes, definitely still not fixed in iOS 10!

I think iOS is more important to become fixed, on Mac it's possible to use other browsers (e.g. show a screen like 'your browser is bad, please use another one'), on iOS this is not possible...

[Daniel Rossi]

Apologies to use this as a forum. 

are you kidding ? 

So they forked the code and fixed it in an OS and browser nobody even has yet ? 

Well that is useful. There should be a patch going out because it's broken right now. Well it has been broken for years especially for canvas captures.

Why not merge it back into webkit ?

[Eric Carlson]

(In reply to comment #55)
> Why not merge it back into webkit ?

All of the work was done in WebKit (153669 and others). This doesn't work on iOS, yet, because some of the lower level frameworks WebKit uses don't have the functionality required.

[Zeno Crivelli]

Regarding fix on iOS, do you think there's a chance it will land in the final iOS 10 release, or we are looking at iOS 11?

[Daniel Rossi]

It needs inline play back support to make it useful which I believe has been made available in 10.  The show stopper as they mentioned is working around some framework constraints or waiting for them to be fixed ? It kind of needed to be implemented years ago so none of this would have been a problem now. 

Have to sit and wait. From my understanding they reckon this has been fixed in nightly webkit. ? I will try and have a look as it also affects canvas image captures not having CORs support.

[Daniel Rossi]

Not sure where that has been changed. I just had a look at the latest nightly. 

Still no crossorigin attribute supported so not even canvas captures will work. Requires the proxy hack.

[Eric Carlson]

(In reply to comment #59)
> Not sure where that has been changed. I just had a look at the latest
> nightly. 
> 
> Still no crossorigin attribute supported so not even canvas captures will
> work. Requires the proxy hack.
>

As I noted above, the WebKit changes depend on features in lower level frameworks.

[chris]

People watching this thread might be interested in another issue here: 

  https://bugs.webkit.org/show_bug.cgi?id=153588

At unpredictable times, safari only renders the first video frame to a canvas, regardless of seek position If the first frame is black then canvas is painted black and remains that way.

Even if *this* issue here is resolved (135379), issue 153588 makes it impractical to use video with canvas. That issue is also listed as P2 and assigned to Nobody.

[Daniel Rossi]

That has nothing to do with this issue I'm afraid. Best not to deviate. That requires a further hack by drawing twice. 

It is really funny but IE11 also suffers the CORS problem but with mediasource appending it doesn't require the proxy hack to draw and get the data from canvas. So for dash it is fine.  

Safari is not. It requires a secondary dash stream to seek and capture via the CORS proxy.

[Daniel Rossi]

Just a heads up about this crazy issue. I can't test VR mode on IOS yet unless I deploy to my Ipad. 

Cordova apps use WebView. It has options to play inline so works around that pesky problem. It works around user interaction. This one is the craziest of them all and has been driving me crazy. 

WebView seems to work around CORS completely even if its a local html file running on file://. 

WebGL is working in Cordova apps without the need for the crossorigin attribute. 

This is both shocking and good news. It won't need the proxy hack but Safari IOS/OSX will. 

I can't check pseudo VR orientation controls but touch controls is working so is  the old cardboard stereo effect. 

I'll have to properly deploy to make it works outside the emulator hopefully.

[Rik Heijdens]

This is also a blocker for JW Player 360 video playback on iOS/Safari.

[Daniel Rossi]

Hi rick I didn't know they had a html5 solution yet or is that the case now ? I provide a Flowplayer alternative. 

I commented on another bug ticket. 

The CORS is still a problem. but even working around that with the proxy serrver hack, HLS now does not render in WebGL. 

That has become an issue recently and other people have picked it up asking questions. I am 100% certain testing before it was working. Going crazy. 

No wonder nobody can support Safari.

[Daniel Rossi]

Yes you are trying to use HLS and will run into the same WebGL rendering issue. Still trying to hunt down a "solution" now. You can try an Mp4 for now over a reverse proxy path and it will work in Safari, IOS but WebView apps completely avoid CORS restrictions. 

There is also now a feature detection issue, Safari is reporting that the crossOrigin attribute is available but still broken. You have to detect for both Safari and Trident browsers and switch to proxy urls. 

I have an example of it live but I need to fix this feature detection break first. 

I am presuming all of this will be resolved in macOS not earlier. IOS 10 will have the CORS issue still as far as I am aware but have inline video playback. A royal mess if you ask me.

[Rik Heijdens]

(In reply to comment #66)
> Yes you are trying to use HLS and will run into the same WebGL rendering
> issue. Still trying to hunt down a "solution" now. You can try an Mp4 for
> now over a reverse proxy path and it will work in Safari, IOS but WebView
> apps completely avoid CORS restrictions. 
> 
> There is also now a feature detection issue, Safari is reporting that the
> crossOrigin attribute is available but still broken. You have to detect for
> both Safari and Trident browsers and switch to proxy urls. 
> 
> I have an example of it live but I need to fix this feature detection break
> first. 
> 
> I am presuming all of this will be resolved in macOS not earlier. IOS 10
> will have the CORS issue still as far as I am aware but have inline video
> playback. A royal mess if you ask me.

Reverse proxying is not an option for us at this point of time.

I'm still having CORS issues with Safari 10 on macOS Sierra:

texImage2D:
SecurityError (DOM Exception 18): The operation is insecure.

Chrome, Firefox and Opera work fine.

[Daniel Rossi]

hahaha so something that was supposed to be fixed in macOS isn't ? 

Here is the deal breaker for HLS. I am so certain it was working for me. Someone is reporting it is not working for them in Safari 9 but I now have it rendering but in Safari 10 but not 9. 

It certainly does not work in IOS but on OSX Safari 10 it does. 

Some kind of regression or sabotage if you will changes have happened I have noticed then. 

A recent update to IOS has now officially broken the proxy hack to stop it working. Says source not supported. 

If people want Apple device support they need the proxy hack but it seems they've gone to great lengths to kill it off completely.

[Tim Pham]

Hi Daniel Rossi,

You mentioned in your comment on 2016-07-06 08:45:36 PDT that you were able to make VR work using Cordova on iOS. Can you please elaborate on how you did that? I am trying to make a web view app that plays inline html5 360 video.

I really appreciate your help!!

Regards,
Tim

[Daniel Rossi]

You don't need the CORS proxy url for Cordova based apps. For some reason it is completely ignored. 

It can play inline , you just need the inline attribute configured on the video element and then configure inline playback configs in Cordova. 

It is only working with mp4 files. 

I am now having webgl rendering issues with HLS that I'm trying to resolved. It's not pretty. Webkit is pretty buggered up really and unstable for webvr. 

I saw the CORS issue becoming a problem years ago trying to implement snapshot features and it was never fixed of course.

[Edwin]

Hi Daniel Rossi,

I'm trying to make HLS work with WebGL in iOS Safari, but still no luck for now, it's driving me like crazy. How's your progress, any luck on solving this issue? 

Regards,
Edwin

[Daniel Rossi]

Edwin I have made a bug report here but zero response. 

I have eventually figured out a work around for OSX Safari if using three.js. They are using this flag that is referred in this ticket that needs to be turned off. 

WebGL seems to render textures in reverse and needs to be flipped around. This flip flag for now reason at all causes the issue. The UV property on the geometry needs to be reverse. 

I tried absolutely everything and nothing. It was working that is for sure. It has been sabotaged. I will keep trying to find work arounds which is standard with Apple. 

https://bugs.webkit.org/show_bug.cgi?id=163866#c3

[Daniel Rossi]

Still requires the CORS proxy of course because of this CORS nightmare issue.  As a reduction method  I will try HLS on the same host without CORS proxy and see what happens.

[Daniel Rossi]

All these problems together is why the big end of town doesn't support Safari / IOS at all. 

I prefer not to use apps though like many don't and stick with WebVR / Html5 streaming. which on IOS would be HLS especially for live streaming.

[Edwin]

Hi Daniel,

I've tried your demo:
http://dev.electroteque.org/threejs/webglworking.html
and it seems worked in iOS 10.1.1 safari with image rendered but flipped.

Regards,
Edwin

[Daniel Rossi]

Edwin is it possible to comment here ?

https://bugs.webkit.org/show_bug.cgi?id=163866#c3

OK so the fixes may work on IOS 10 not IOS 9. 

The actual Three.js variant fix is here, the texture is flipped via the mesh instead of in the shader program which happens per frame. 

http://dev.electroteque.org/threejs/hls.html

This ticket is about CORS issues.

[Daniel Rossi]

Apparently the CORS issue is fixed in Safari in macOS but the very latest updates. The inline video playback is also working in IOS. 

Here is two tests on for mp4 streaming another for HLS. The HLS one requires extra work because of a separate bug with FlipY that is still a problem on OSX Safari. 

HLS rendering on IOS 10 is displaying but has colour artifact issues. The frames stop working but I believe its an issue with the emulator and frames dropping. I now have no device that can be updated to IOS 10.  It doesn't show up at all on IOS 9. Both require the CORS proxy for mp4 and HLS. 

http://dev.electroteque.org/webgl/threejscors.html
http://dev.electroteque.org/webgl/threejscors-hls.html

[Daniel Rossi]

Now I'm stuck with a situation that the crossOrigin feature detection cannot be used because it reports of support in OSX 10.11 but there is no such thing. 

I have to try and detect which OSX safari is used in also. very bad. 

var testVideo = document.createElement("video");
 testVideo.crossOrigin = "anonymous";
testVideo.hasAttribute("crossOrigin")

This used to work but now it doesn't. Safari in 10.10 / 10.11 will report the attribute is available.

[Daniel Rossi]

Here is some Es6 from static utils to deal with the CORS detection problem. It has driven me over the edge. I know people have given up on Safari but people want and need this to work.  

Both IOS 10 Safari and Yosemite Safari 10 report CORS feature detection but obviously do not, this was a recent sabotage in Safari 10. 

Extra platform and version checks are required. This is rough as guts but I have no time or stomach to fine tune it further. If it doesn't support CORS the property should not be available as before. 

I have to specifically check for both Safari and OSX 10.12 and above. This will break in the future of course. 


static supportsCORS() {
        let testVideo = document.createElement("video"),
            hasCORS = false;

        testVideo.crossOrigin = "anonymous";
        hasCORS = testVideo.hasAttribute("crossOrigin");
        testVideo = null;

        if (hasCORS) {

            if (WebVRUtils.isSafari) {

				if (WebVRUtils.isIpad) return false;
                return WebVRUtils.isNewSafari;
            }

            return true;
        }

        return false;

    }

    static get isSafari() {
		const userAgent = navigator.userAgent;
        return (/Safari/i).test(userAgent) && !(/Chrome/i).test(userAgent);
    }

	static get isIpad() {
		const userAgent = navigator.userAgent;
        return (/iP(hone|od|ad)/i).test(userAgent);
    }

    static get isNewSafari() {
        const version = /Mac OS X (10[\.\_\d]+)/.exec(navigator.userAgent)[1].split("_")[1];
        return +version >= 12;
    }

[Kieran Farr]

Confirmed this works as expected (bug fixed) when testing against the URL on this ticket using latest macOS Sierra 10.12.1 (16B2659) Safari Version 10.0.1 (12602.2.14.0.7).

Bug is still NOT fixed in Mobile Safari on iOS 10.2 Public Beta 3 (14C5077b). There is a newer Beta 4 available but I have not yet installed or tested.

[Esteban]

Any news on this bug fix for iOS? Currently this is blocking us from deploying 360 video to production using iOs and videos hosted in CDN

[Daniel Rossi]

IOS 10 still not fixed. IOS 10 required framework changes I believe. Which they are fixing underlying issues first ?

[mrosellini]

Any update or ETA when this fix will get into iOS Safari? That seems to be the lone holdout and even once an update is pushed we have to factor adoption time leaving most publishers in the situation of having to setup or keep doing the proxy workaround...

[Jer Noble]

(In reply to comment #83)
> Any update or ETA when this fix will get into iOS Safari? That seems to be
> the lone holdout and even once an update is pushed we have to factor
> adoption time leaving most publishers in the situation of having to setup or
> keep doing the proxy workaround...

We will absolutely update this bug as soon as there is anything to announce.

[mrosellini]

(In reply to comment #84)
> (In reply to comment #83)
> > Any update or ETA when this fix will get into iOS Safari? That seems to be
> > the lone holdout and even once an update is pushed we have to factor
> > adoption time leaving most publishers in the situation of having to setup or
> > keep doing the proxy workaround...
> 
> We will absolutely update this bug as soon as there is anything to announce.

Thanks Jer, we also filed a bug report with bugreport.apple.com, ticket 31066138. Do appreciate anything that can be done to get a fix into the next update.

I have been waiting for a long time for this bug to be fixed.

[Madjid Taha]

Hi guys,

Lately I got that problem at work and I found a client-side workaround waiting for this bug to be fixed so I decide to write an article about it.
I hope it can help some of you.
https://blog.madj.me/safari-ios-problems-when-drawing-video/

[Daniel Rossi]

You would have to wait for the entire video to finish no ? I guess that wouldn't work with HLS videos either so live streaming. 

It has certainly caused problems for alot of people. The only work around for IOS is inside a Cordova app. 

Hopefully what I just said means Apple don't try and sabotage and block it. It doesn't do any CORS check whatsoever. Works with Mp4 and HLS. 

Please Apple and don't try and block that now ! 

This bug has been a problem for half a decade or more. It's just only that people are wanting to do 360 videos and webgl video textures it has come back to bite them. 

This is not a problem anymore on macOS. Have a look.

[Daniel Rossi]

Just make a player as you would any player and build inside a Webview Cordova / Ionic app. 

Does not need CORS proxy, no crossorigin attribute , any source. 

I believe pinned WebView apps also do not use CORS as far as I tests go.

[Jon Lee]

*** Bug 170455 has been marked as a duplicate of this bug. ***

[Sergei Soloviev]

Hi! Any planning updates here? We have the same problems with showing videos as textures in Canvas.

[Jer Noble]

(In reply to Sergei Soloviev from comment #91)
> Hi! Any planning updates here? We have the same problems with showing videos
> as textures in Canvas.

I strongly urge everyone who's interested in this feature to sign up with the developer.apple.com portal and try it out with iOS 11 Beta.

[Daniel Rossi]

So IOS10 no good. The only thing IOS10 provided was inline playback. 

The fix is in IOS11 but won't be ready for months ? 

My subscription ended sadly, I don't need it at all. 

Underlying frameworks prevent the issue being rolled out for IOS1- then ? 

Its getting a bit much having to ask people to implement the CORS proxy workarounds that dont scale.

[chris]

(In reply to Jer Noble from comment #92)
> (In reply to Sergei Soloviev from comment #91)
> > Hi! Any planning updates here? We have the same problems with showing videos
> > as textures in Canvas.
> 
> I strongly urge everyone who's interested in this feature to sign up with
> the developer.apple.com portal and try it out with iOS 11 Beta.

Your response is vague. Will this three-year old bug be resolved or not? When?

[Jer Noble]

(In reply to chris from comment #94)
> (In reply to Jer Noble from comment #92)
> > (In reply to Sergei Soloviev from comment #91)
> > > Hi! Any planning updates here? We have the same problems with showing videos
> > > as textures in Canvas.
> > 
> > I strongly urge everyone who's interested in this feature to sign up with
> > the developer.apple.com portal and try it out with iOS 11 Beta.
> 
> Your response is vague. Will this three-year old bug be resolved or not?
> When?

We believe it will be fixed in iOS 11. But interested parties (e.g., you) should try out your test cases against iOS 11 to verify (one way or another) whether it fixes the issues you're seeing.

Apple has released a public beta for iOS 11, which does not require a developer account: <https://beta.apple.com/sp/betaprogram/>.

[Klaus Reinfeld]

In the latest iOS 11 beta 4 from 24. July 2017 it is still NOT working!

[Daniel Rossi]

"try it out yourself" 

it either is committed and reported as fixed in this ticket or not. 

Seems like it's still has not been fixed. This has been a problem for years. I've been needing to use CORS proxies for canvas capture features. 

It only became a bigger issue once people started to try and use webgl video textures !

[Daniel Rossi]

According to the specs. like IOS 10. IOS 11 won't run on anything newer than a 2017 Ipad. Requires a new device.

[Daniel Rossi]

I just tried IOS 11 in Xcode 9 beta and CORS is still not there it seems. Not sure how to get the specific IOS 11 simulator version.

[Jer Noble]

(In reply to Daniel Rossi from comment #99)
> I just tried IOS 11 in Xcode 9 beta and CORS is still not there it seems.
> Not sure how to get the specific IOS 11 simulator version.

Looks like this regressed in general, because the same site now fails in Safari Tech Preview on Desktop as well.

[Daniel Rossi]

Wow. 

macOS Safari in production definitely had CORS fixed. 

At the same time as the IOS 10 release CORS wasn't fixed in that sadly. 

Detecting it was a problem though, it requires platform version checks. It returns true when checking if the crossorigin attribute is set. Previously it's empty.

[Jer Noble]

(In reply to Daniel Rossi from comment #101)
> Wow. 
> 
> macOS Safari in production definitely had CORS fixed. 
> 
> At the same time as the IOS 10 release CORS wasn't fixed in that sadly. 
> 
> Detecting it was a problem though, it requires platform version checks. It
> returns true when checking if the crossorigin attribute is set. Previously
> it's empty.

The test is failing on desktop because the HTMLMediaElement::hasSingleSecurityOrigin() is failing; and that's failing because the media element url was redirected from http: to https:. Can you try this test without the redirection, or is it possible the CDN changed policies to do the redirect to https:?

[Jer Noble]

(In reply to Jer Noble from comment #102)
> (In reply to Daniel Rossi from comment #101)
> > Wow. 
> > 
> > macOS Safari in production definitely had CORS fixed. 
> > 
> > At the same time as the IOS 10 release CORS wasn't fixed in that sadly. 
> > 
> > Detecting it was a problem though, it requires platform version checks. It
> > returns true when checking if the crossorigin attribute is set. Previously
> > it's empty.
> 
> The test is failing on desktop because the
> HTMLMediaElement::hasSingleSecurityOrigin() is failing; and that's failing
> because the media element url was redirected from http: to https:. Can you
> try this test without the redirection, or is it possible the CDN changed
> policies to do the redirect to https:?

To be clear, this part of the code hasn't changed in about 3 years, so it's unlikely that this test regression was caused by a change in WebKit.

[Daniel Rossi]

Yes that is a standard procedure. CORS won't work over http / https. Has to be the same protocol. So that is a non issue. It's just IOS everyone is waiting on. 

I just tested again and Safari 10.1.2 and mac OS 10.12.6 is perfectly fine. 

If you mean 3 years since macOS then yes it was when that was fixed. 

You can see on my own feature and demos. same ssl protocol for site and streams on cloudfront. 

https://flowplayer.electroteque.org/vr360/fp6

[Jer Noble]

(In reply to Daniel Rossi from comment #104)
> Yes that is a standard procedure. CORS won't work over http / https. Has to
> be the same protocol. So that is a non issue. It's just IOS everyone is
> waiting on. 
> 
> I just tested again and Safari 10.1.2 and mac OS 10.12.6 is perfectly fine. 
> 
> If you mean 3 years since macOS then yes it was when that was fixed. 
> 
> You can see on my own feature and demos. same ssl protocol for site and
> streams on cloudfront. 
> 
> https://flowplayer.electroteque.org/vr360/fp6

Daniel, your site seems to work fine on iOS 11 (hardware, not simulator). Can you confirm?

[Daniel Rossi]

It's using a CORS Proxy work around feature as fallback.  

What source is it using ? 

this is the cors proxy

//flowplayer.electroteque.org/video/360/ultra_light_flight_720p.mp4

I don't have access to a Device that can run IOS11 yet I am afraid, my focus has been Android VR. I need a brand new device there.  

I only have access to IOS 11 Simulator inside Xcode 9 beta.

[Daniel Rossi]

I made a few extra examples from previous examples. 

hopefully these work as I was unable to get click and video playback in the simulator. one is the cors proxy. 

Works on Android. 

https://dev.electroteque.org/safari/example/
https://dev.electroteque.org/safari/example/proxy.html

[Jer Noble]

(In reply to Daniel Rossi from comment #107)
> I made a few extra examples from previous examples. 
> 
> hopefully these work as I was unable to get click and video playback in the
> simulator. one is the cors proxy. 
> 
> Works on Android. 
> 
> https://dev.electroteque.org/safari/example/
> https://dev.electroteque.org/safari/example/proxy.html

The <video> element in these examples needs the “playsinline” attribute in order to not play in full screen mode on iPhone, but the non-proxy example seems to work correctly on iOS 11.

[Daniel Rossi]

OK. Ive added the new spec flag which is in production, those demos came from threejs so has the old flag. 

The IOS11 simulator has problems and wont even start playing video. I get a frame up on IOs10 simulator with the proxy. 

So regarding this ticket there is a possibility IOS 11 beta on a device not a simulator has CORS fixed. 

I have to somehow get a new device that supports IOS11 to properly test and confirm further thanks.

[Donni]

I just tried the iOS 11 GM and the issue seems to persist.

[MooLee]

Ios11 has solved the cross problem



http://res7.utovr.com/player_src.html


http://res7.utovr.com/player_src.html?src=http://cache.utovr.com/201508240527049953.mp4

[cjl]

(In reply to MooLee from comment #111)
> Ios11 has solved the cross problem
> 
> 
> 
> http://res7.utovr.com/player_src.html
> 
> 
> http://res7.utovr.com/player_src.html?src=http://cache.utovr.com/
> 201508240527049953.mp4

CROS problems really fixed in ios11. This UtoVR sample solved ios11 hls FLIP_Y problem，but the code is mixed by UtoVR， could you tell us how to fixing FLIP_Y problem. thanks!

[Daniel Rossi]

Confirming CORS is fixed in IOS 11.1. I just finally got the budget to get a hardware device for testing. 

There is no need for CORS proxy with 11.1. I don't think CORS attribute support can be properly tested with Safari. Have to do client version checks. 

There is major HLS/webgl regression issues to go through now though.

@Daniel Rossi
I can't seem to run VRView from the Cardboard SDK on iOS 11. Any pointers?
https://codepen.io/anon/pen/VrRBNg

[Daniel Rossi]

I haven't checked WebView under IOS11 as in a Cordova app. 

But as before WebView didn't even factor in CORS at all, it was working. I hope that is not broken too now ! 

I have to check myself. 

I believe this is possibly what you are talking about.

@Daniel Rossi
The VRView in question is HTML5.
https://developers.google.com/vr/concepts/vrview
https://codepen.io/anon/pen/VrRBNg

Could you have a look?

[Kimmo Kinnunen]

If I understand the comments correctly, the use case now works.
I'll close this as fixed.
Please file a new bug report if there are new problems around this area.

[Klaus Reinfeld]

This is not fixed, the bug is still there and still annoying!

Just try the link from the initial post on any iOS device:

http://krpano.com/ios/bugs/ios8-webgl-video-cors/

[Klaus Reinfeld]

Sorry for the last post, I was wrong, it seems to work now.

[Daniel Rossi]

I've had nobody complaining of CORS bugs. It was Fixed in IOS 11.1. Ive not seen a problem. Is this still happening ? HLS rendering has always been patchy and unstable however.