Support WebRTC in Safari in-app browser

• Component: WebRTC
• Status: RESOLVED FIXED

[Dag-Inge Aas]

Filing a bug since I couldn't find another bug to track progress on this. WebRTC is currently not supported in Safari in-app browser, which is making the end-user experience of going to url-based video conferencing platforms like appear.in or Confrere hard to use. 

An example is that a user receives a link on e-mail or in Slack. User clicks the link and is taken to Safari in-app version of the browser. getUserMedia is not supported here, so call fails.

We want getUserMedia and other WebRTC related APIs to work in Safari in-app as well as WKWebView (I believe this is what Facebook uses?) so that WebRTC APIs may also work here.

Alternatively, an escape hatch has to be made available to open a link in "proper" Safari where we can use WebRTC APIs. Currently I have found no way of achieving this. In our user testing, regular users have difficulty finding or understanding why they have to click the little "Open in Safari" icon to make video calling work. Facebook makes matters worse by hiding this option behind a kebab menu, but to be fair, they also don't support WebRTC on Android inside Facebook Browser.

Steps to reproduce:
- Send yourself the following link in an application which triggers the in-app browser: https://webrtc.github.io/samples/src/content/getusermedia/gum/
- Observe that getUserMedia does not complete, and the frame remains black. No feedback to the user is given of this error.

[Radar WebKit Bug Importer]

<rdar://problem/37987846>

[Radar WebKit Bug Importer]

<rdar://problem/37987852>

[youenn fablet]

Hi Daginge,

This is a known limitation.
PeerConnection should work but not yet getUserMedia unfortunately.

[Dag-Inge Aas]

Thanks youenn! Is there any public timeline for when this will be supported? I suspect it's a security & UX issue rather than technical limitations at this point. Support for WebRTC in the in-app browser could be critical to our product, so naturally we are very focused on this issue.

For what it's worth, these past few patches we've seen great improvements in the WebRTC platform on iOS, and are now more and more comfortable letting "regular" end users use it on a daily basis. But the link issue opening randomly in standalone Safari and sometime in-app (as seen from the user's perspective) is making the end user experience very confusing still.

[Dag-Inge Aas]

Just bumping this. Will santa give me this for iOS 12? :) It's a huge limitation because most e-mail apps and calendar apps now use the in-app version, meaning we have to have ugly UI to make people click the Safari button...

[Ben]

@daginge can you explain how you detect in-app browser and switch to Safari?

[Dag-Inge Aas]

We only detect Safari in-app, we aren't able to automatically upgrade, so we have to direct the user to do so. We detect in-app by saying: If we are on an iOS device, and the Safari version SHOULD support WebRTC, but doesn't, then we can assume it's an in-app browser and we render a specific message telling the user to click to go to the full version of Safari.

Again, it would be a MUCH better user experience if at least the in-app version of Safari supported WebRTC in the near future...

[Dag-Inge Aas]

Any news on support in Safari in-app for the next release?

[Raship Shah]

+1 for this feature

+1

[Sebastian]

+1

[tim]

+1 for this change, we have the same problem with IOS 12 here, we don't have any WebRTC support inside in app browsers.

Same here (ios 12.1), cannot make the camera work when using HTML5 manifests. It's a big problem for us as we can't distribute our app as expected. Thanks.

[tim]

just send a simple like to: https://webrtc.github.io/samples/ over hangouts for example and click in it.
None of the demo's would work.

You can also try https://appr.tc/ which won't work also (IOS 12.1).

can we get some response from the Dev team here? all examples are working in in-app Android browsers.

[tim]

it looks like turn and stun servers are failing to run onicecandidate on every WKWebView.

We even tried Chrome and Firefox for IOS but since they are both built on WKWebView, they also don't work.

We managed to get WEBrtc to work only with Safari when it is NOT an in app browser mode.

[youenn fablet]

navigator.mediaDevices.getUserMedia is not yet exposed/working in SafariViewController/Web.app and WKWebView.

RTCPeerConnection should be working though.
I am surprised that STUN/TURN servers are not working properly.

+1

[tim]

(In reply to youenn fablet from comment #16)
> navigator.mediaDevices.getUserMedia is not yet exposed/working in
> SafariViewController/Web.app and WKWebView.
> 
> RTCPeerConnection should be working though.
> I am surprised that STUN/TURN servers are not working properly.

we've tried both turn and stun servers, no matter what we do, in an in-app mode onicecandidate is never fired.

we manage to setLocalDescription and setRemoteDescription and that's it, nothing happens.

on Safari, and any other android/in app android browser it is working.

[tim]

any news regarding this from an official webkit dev?
if you all say that turn and stun should work in an in app browser we would love to see a live example, the same code that is working on Safari isn't working inside wkWebView

[youenn fablet]

(In reply to tim from comment #19)
> any news regarding this from an official webkit dev?
> if you all say that turn and stun should work in an in app browser we would
> love to see a live example, the same code that is working on Safari isn't
> working inside wkWebView

I tried loading https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ in a WKWebView using https://itunes.apple.com/us/app/webview-wkwebview-and-uiwebview-rendering/id928647773
I was able to gather a srflx candidate.

Note that by default, host candidates are not exposed.
Note also that if a connection is created using a peer reflexive candidate, there might be no srflx candidate being exposed at all.

Tim, could you create a repro case?

[Jaroslav Bauml]

+1

[Takahiro Ichihashi]

Youenn 
Hello, do/can you see a chance to discuss this problem regarding next milestone (maybe Safari 13)? I noticed this compatibility issue recently and potential impact to the ecosystem seems enormous... :)

[Dan]

+1

[Jason DeVries]

+1 On getting this finally implemented sooner rather than later. Difficult to showcase speech-to-text functionality on a website when a lot of advertising is social media based which usually results in users trying to visit the website in an in-app browser session. If it doesn't work... doesn't grab their attention in the first second, they'll likely just close it rather than reading they need to open in Safari.

[Kara]

I would also love to know the status of this. Surprised it hasn't been prioritized higher given that a significant amount of traffic is through the in-app browser. Is there a plan to get it added in an upcoming release?

[Stefan Sechelmann]

+1

[Alojz Milicevic]

+1

[Dag-Inge Aas]

For everyone following this bug, I'm happy to say that Safari in-app (SFSafariViewController) now supports getUserMedia in iOS 13, which in turn means that full WebRTC support has been reached.

To feature test between old and new Safari in-app, check for the presence of RTCPeerConnection and navigator.mediaDevices.getUserMedia. If only RTCPeerConnection is present, it should be Safari in-app in iOS 12 or earlier. If both are present, you are in Safari in-app (well, Safari since at that point there's no way of distinguishing them) on iOS 13+.

Rejoice!

(But no, still no support in WKWebView, which means Chrome/Firefox/Other browsers on iOS are still being blocked from using webcam)

[Maciej Stachowiak]

Marking this as resolved since SFSafariViewController now supports getUserMedia. Might be worth filing separate bugs for other WKWebView clients.

[Mike]

For those that were tracking this specifically for WKWebView and not SFSafariViewController, you may want to start watching #188360:
https://bugs.webkit.org/show_bug.cgi?id=188360

[Noah Mehl]

I know this is a bug for webkit, but many people would like this functionality.

In the mean time we wrote an open source swift shim for the missing WebRTC functionality for WKWebView:

https://github.com/OpenTelecom/WKWebViewRTC

Big thanks to the Cordova project!

(In reply to Noah Mehl from comment #31)
> I know this is a bug for webkit, but many people would like this
> functionality.
> 
> In the mean time we wrote an open source swift shim for the missing WebRTC
> functionality for WKWebView:
> 
> https://github.com/OpenTelecom/WKWebViewRTC
> 
> Big thanks to the Cordova project!

It works like a charm, thank you so much!

[zn]

This looks promising. Now two questions:

Does this implementation allow the audio from a WebRTC session to mix with background audio from the iOS app? When using SFSafariWebViewController this is not the case and the browser ducks everything else. 

Second, I wonder if this implementation can be ported to Obj-C so it can be more easily ported to Xamarin projects. 

Great find nonetheless!!!

[Noah Mehl]

(In reply to molnar.timi7 from comment #32)
> (In reply to Noah Mehl from comment #31)
> > I know this is a bug for webkit, but many people would like this
> > functionality.
> > 
> > In the mean time we wrote an open source swift shim for the missing WebRTC
> > functionality for WKWebView:
> > 
> > https://github.com/OpenTelecom/WKWebViewRTC
> > 
> > Big thanks to the Cordova project!
> 
> It works like a charm, thank you so much!

You're very welcome!  If you find anything or think the project could use some more documentation, please open an issue in Github or submit a PR :)