Bug 208988 – [WebXR] Implement WebXR device API

• Component: WebXR
• Status: NEW

[Sergio Villar Senin]

This is a metabug to track progress of WebXR device API implementation

[Artem Batogovsky]

Hi!

Can we test it already? We want to test WebXR on iOS with out product as soon as possible to ensure that our users will have best possible experience

[Will Walker]

Will the WebXR Device API be supported in Safari 15?

[Shashank Agrawal]

Any further update here? I see that this now depends on only two tasks that are pending.

[oladam]

When can you expect the implementation to be completed? I'm sure many developers out there are eager for this to happen

[Sam]

Please try to complete this ticket before 2050

[Pierre Coulet]

Hi, is there anyone with the knowledge to enlighten us on an estimated date for availability?

[David Morehead]

Yes, very interested in the progress of this issue as clients "can't believe" that webxr is not yet supported on IOS.

[Frannie Brennan]

Still very keen to see this :)

[Al Murray]

Not to clog the comments, but everyone and their mother is waiting eagerly for this. Whole new ballgame on the AR front.

[Zach Rattner]

Is there any update on an ETA for this feature, or anything an external party could do to help?

[Nik Lever]

Any news on this? Lots of developers waiting for WebXR on iOS. Fingers crossed it lands this year.

[Sam Sneddon [:gsnedders]]

<rdar://83079238>

[Robert Farthing]

Any ETA on this?

Any update / ETA?

I know there has been rumours of support for WebXR in iOS16 for June 2022

Can anyone in the know on this area confirm anything?

[Tavius Koktavy]

Excited for the possibilities here!

I see the beta flags but they don't seem to enable WebXR on iPhone 12 Pro 15.6.1, has anyone had success?

Is there maybe something needed in JavaScript to tell WebXR that it should work on Safari?

[Kevin Staunton-Lambert]

Hi there, just curious if I need a cert to be able to run immersive-ar session? I’d like to run the WebXR emulator too to support immersive-vr, is this possible with the flags set?

[Daniel Rossi]

I've enabled all the WebXR flags in IOS 16.4 and these samples failed to detect for WebXR. should it display a cardboard mode like native WebXR in Android ? Do we have to wait to test until IOS 17 which makes some devices redundant and won't be able to get IOS 17. ?

I've had complaints to reinstate the WebXR polyfill but the project is vacant and its completely broken. Unrelated to IOS but I had to fix fullscreen support as iPhone has no native fullscreen support to show the canvas ! That is another oversight no html fullscreen make the canvas fullscreen. The fullscreen hack for iPhone is a fixed position 100wh 100vh css rule. 

Is there a way to check a build that actually has WebXR enabled ?

https://immersive-web.github.io/webxr-samples/

this check should work if WebXR is enabled. 

https://jsfiddle.net/danrossi303/ng7x4aw3/3/

More info about the polyfill broken still needed for IOS. 

https://discourse.threejs.org/t/webxr-polyfill-broken-with-latest-three-js/51812/6

[Daniel Rossi]

I was able to check on an Ipad again. xr test is detected. But the next step is checking what type it supports. I believe the session support check may be failing. This should show if its supported.

I get a promise failure with "the operation is insecure". I'll keep testing. 

https://jsfiddle.net/danrossi303/ng7x4aw3/9/

[Daniel Rossi]

Can't edit comments. It seems it fails within an iframe with a security error so jsfiddle wont work. So new check. so sessions are not supported. I tried both immersive-vr and inline. 

https://electroteque.org/dev/xr/xrcheck.html

[Daniel Rossi]

Created attachment 466534 [details]
Android Chrome Native WebXR

[Daniel Rossi]

Will sessions be supported without a headset like Android Chrome has native support for ?. Is that why session support checks are false in IOS 16.4 ?

[Daniel Rossi]

I'm sorry I can't update comments. One thing of importance with WebXR. Landscape screen locking api. IOS doesn't have this, Android does. It requires a css orientation hack like iPhone html fullscreen. Will IOS be getting screen orientation lock api ? 

https://developer.mozilla.org/en-US/docs/Web/API/ScreenOrientation/lock

[Daniel Rossi]

I've made a new ticket regarding screen orientation lock api which is needed for both fullscreen canvas/video content and WebXR. It's been available in Android for 6 years. 

https://bugs.webkit.org/show_bug.cgi?id=257695

There is already a ticket regarding to enabling html fullscreen support for iPhone, where Ipad has fullscreen support. It still won't support launching canvas elements fullscreen. Css fullscreen hacks just don't work well, and landscape lock css hacks work for the Iphone hack but not the best either. You can't manipulate orientation of Ipad fullscreen elements however, so in portrait mode it looks silly. 

https://bugs.webkit.org/show_bug.cgi?id=206854

I had to work on a hack to launch canvas rendering into a seperate video and make that fullscreen for iPhone. I'm surprised it actually worked and managed to hack it to autoplay, not making the video inline allows it to play and show properly. I think there is a bug with muted and inline video with MediaStream. I worked around the clock to pull it off. Device orientation of the original rendering is reflected on the second fullscreen video also. 

https://github.com/danrossi/canvas-pip-fullscreen/blob/main/src/CanvasFullscreen.js#L26

I think both are needed at the same time.

[Daniel Rossi]

I've yet to upgrade to WebGPU myself. But 6 years later it's looking likely things are moving towards WebGPU from WebGL. It's looking like different methods are required for an XR session and layers. It might be good to have WebGPU support also with XR if not already. Along side html canvas fullscreen support and therefore landscape orientation lock with html fullscreen support. 

https://github.com/immersive-web/WebXR-WebGPU-Binding/blob/main/explainer.md

I've been reading several unresolved threads regarding video playback failures with WebXR (example thread here: https://discourse.threejs.org/t/video-texture-no-longer-updating-after-entering-webxr-mode/43068).
Has this been addressed or is it on the radar and road mapped already? Is this an appropriate place to raise the issue?

[Daniel Rossi]

Those are unrelated to webkit WebXR. I'm hoping there is a native view like Chrome has on Android or does it need a headset ? I have so many people asking about it and the polyfill is deprecated and broken. Has WebXR actually been integrated into IOS for testing yet. As it wouldn't accept sessions with my fiddle test code.

WebXR has only been integrated into VisionOS, not iOS.

Daniel, how are those unrelated to webkit WebXR?
The separation between WebXR Device API and Webkit is a bit blurry to me.

[Kevin Staunton-Lambert]

Well for AR on phones/tablets I feel it’s important too - also video projectors are increasingly being used for real world interactive entertainment/exhibits which could easily be browser/WebGPU based, just other things WebXR in WebKit has value

[Daniel Rossi]

WebXR is not enabled by default in the VisionOS 1.0/1.1 Simulator. In 1.0 launching video to WebXR crashed and prevents video further playing back. In 1.1 with my implementation I am having issue getting WebGL video rendering and web inspection of 1.1 doesn't even work only in 1.0. I can't seem to find a solution for the broken web inspection in 1.1. 

Can try and launch this in 1.0 and see the crash. It goes to WebXR but further playback stops. Video playback stops. I think it's a browser crash but no error. Controls stop working also. Have to refresh the page. 

https://electroteque.org/plugins/videojs/vrvideo/

[Daniel Rossi]

Created attachment 470177 [details]
visionOS webxr session

[Daniel Rossi]

Created attachment 470178 [details]
Chrome webXR Session

displays "enabledFeatures"

[Daniel Rossi]

I lost a whole message adding screenshots. I am needing "enabledFeatures" property missing in the visionOS webXR session to determine hand-tracking support. Hand controls can be added after the session reports hand-tracking features is supported. 

visionOS 1.0/1.1 simulator has no input sources or hand-tracking support. So I am unable to check hand controls or other vr controllers in it. 

I am trying to figure out how to determine this right now due to that missing feature. Check for "MacIntel" / Ipad platform and if there is inputSources ? If vr controls as inputSources is enabled there should be a connected event also. So not sure how to determine this between the simulator and the headset. I don't have a headset to check.

[Daniel Rossi]

Due to the lack of the hand-tracking feature property and lack of inputSources with the simulator something like this I came up with and seems to be cross compatible. So nothing gets added for the simulator. Assuming the headset provides inputSources I can't check. 

https://discourse.threejs.org/t/detect-hand-tracking-xr-feature-support-to-add-hand-controllers/62229/4

[Daniel Rossi]

Since Apple Vision OS 1.1 simulator, a mouse click acting as pinch turns on a "transient-pointer" input source. And the targetRaySpace is updated on movement. I can get raycasting of objects working. webinspect is broken so I had to build a logging proxy to see what was actually going on.

I wish a native WebXR was made available for IPhone like Android on top of native html fullscreen. I thought that is what was happening. I'm forced to try and implement a work around and is not easy.

[Wladislav Artsimovich]

I'm deep in a WebXR project and quite sad to see iOS being unsupported.
Happy to support with any kind of testing.