Bug 229752 - [WebXR] Support Raw Camera Access in WebXR
Summary: [WebXR] Support Raw Camera Access in WebXR
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebXR (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL: https://immersive-web.github.io/raw-c...
Keywords: InRadar
Depends on:
Reported: 2021-09-01 00:46 PDT by andj2223
Modified: 2022-05-30 19:35 PDT (History)
3 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description andj2223 2021-09-01 00:46:28 PDT
Raw camera access is important for many XR usecases. The quoted text and source links below explain further.
Though this is currently a draft spec, it would be great to get the ball rolling sooner than later on supporting this in Webkit.

> Granting the camera access to the application could allow the applications to:
> * Take a snapshot of the AR experience along with application-rendered content. This is mostly relevant for scenarios where the camera image serves as the background for application's content (for example handheld AR).
> * Allow the application to overlay some render effects over the entire scene.
> * Provide feedback to the user about the environment they are located in. This may be particularly useful for VR headsets with available cameras (for example WMR headsets and the Flashlight feature). Note: this use case is not addressed by the current proposal - see Immersive Web CG discussions section.
> * Run custom computer vision algorithms on the data obtained from the camera texture. It may for example enable applications to semantically annotate regions of the image, for example to provide features related to accessibility.
> * Remote (server side) rendering where there is a need to sync a remotely rendered result to the camera output.

Spec: https://immersive-web.github.io/raw-camera-access/
Explainer: https://github.com/immersive-web/raw-camera-access/blob/main/explainer.md
A github issue outlining some usecases for raw camera access: https://github.com/immersive-web/raw-camera-access/issues/7
Comment 1 Radar WebKit Bug Importer 2021-09-08 00:47:15 PDT
Comment 2 andj2223 2022-05-30 19:35:12 PDT
Any updates?