Bug 262416 – REGRESSION (iOS 17): getUserMedia Back Triple Camera automatically switches cameras

• Component: WebRTC
• Status: NEW

[Joel Udwin]

Created attachment 467982 [details]
iOS17 Back Triple Camera Regression

This is a regression between iOS 16 and 17. 

It appears that the Back Triple Camera accessed via getUserMedia switches cameras from wide to ultra wide while cropping the media stream to maintain the original field of view. 
This change in behavior is problematic as it impacts computer vision applications on the web, and there is no way to prevent the automatic switching. 

Our testing on devices with iOS 16 reveals that this behavior does not occur prior to iOS 17. 

I've attached a minimal example video of the issue to this report. 

Reproduction Steps:

1) Open: https://webrtc.github.io/samples/src/content/devices/input-output/
2) Allow camera and microphone permissions
3) Select Back Triple Camera as the Video source
4) Move the camera close to objects and then back up 
5) Observe that the camera switches when close to objects v.s when backing away from objects

[Radar WebKit Bug Importer]

<rdar://problem/116407580>

[Kimmo Kinnunen]

Thank you for the report. I could repro the camera view flickering few times initially. I could also repro the camera switch upon close objects, but the switch seems to happen only once.

In general, if I understand correctly the purpose of the triple camera is to switch cameras based on what is being looked at, so that the captured picture would be the best possible of the multiple cameras. 

I agree that there are seams and non-optimal behavior.


> 5) Observe that the camera switches when close to objects v.s when backing away from objects

Can you specify in more detail:

- Is the problem that it only switches once? You'd like it to switch consistently? Or you'd like it to never switch?

- Is the problem that it switches, or that it switches and you're able to tell it switches? E.g. if it was seamless for humans, it'd be ok?

- Is the problem that you are not able to specify getUserMedia constraints that would not give the triple camera? E.g. if you were, you'd select a camera that never switches, and this would solve the problem?

[youenn fablet]

> I could repro the camera view flickering few times initially.

We have a regression in the rendering of camera video frames, are you referring to this? See https://bugs.webkit.org/show_bug.cgi?id=261330.

> It appears that the Back Triple Camera accessed via getUserMedia switches cameras from wide to ultra wide while cropping the media stream to maintain the original field of view. 

I do not think webkit is actually doing anything specific here so this might be below.
Is this different from native app behaviour?

[Joel Udwin]

Can you specify in more detail:

- Is the problem that it only switches once? You'd like it to switch consistently? Or you'd like it to never switch?

The problem is that the camera switches automatically at all. Automatic switching of the Back Triple Camera accessed via getUserMedia is a new behavior that did not occur previously before iOS 17.

- Is the problem that it switches, or that it switches and you're able to tell it switches? E.g. if it was seamless for humans, it'd be ok?

In our case the problem is that switching occurs (given that expectation did not exist in prior versions of iOS. The camera switching unexpectedly impacts the quality of computer vision applications on the web. 

- Is the problem that you are not able to specify getUserMedia constraints that would not give the triple camera? E.g. if you were, you'd select a camera that never switches, and this would solve the problem?
This would help solve the problem. If there was a way to select based on this constraint there would be no impact even though iOS 17 led to a change in behavior on some cameras accessed via getUserMediais.

[Kimmo Kinnunen]

(In reply to Joel Udwin from comment #4)
>> - Is the problem that you are not able to specify getUserMedia constraints
> that would not give the triple camera? E.g. if you were, you'd select a
> camera that never switches, and this would solve the problem?
> This would help solve the problem. If there was a way to select based on
> this constraint there would be no impact even though iOS 17 led to a change
> in behavior on some cameras accessed via getUserMediais.

You could experiment with passing "zoom: false" as a constraint. E.g.:

navigator.mediaDevices.getUserMedia({video: { zoom: false, facingMode: "environment" }})
.then(mediaStream => {
    let vE = document.querySelector('video');
    vE.srcObject = mediaStream;
});