Bug 227933 – applyConstraints recaptures camera and creates a lag

• Status: REOPENED
• Component: WebRTC

[Manik]

When using the applyConstraints API on getUserMedia, it seems like Safari is recapturing from the camera. You will also see a lag in the video rendering each time applyConstraints is called.

I've set up a basic JSFiddle that helps repro this:
https://jsfiddle.net/3jztwxc6/

```

(async () => {

const stream = await navigator.mediaDevices.getUserMedia({audio: true, video: true});
const videoElement = document.getElementById("gum");
videoElement.srcObject = stream;
const videoTrack = stream.getVideoTracks()[0];

const constraints = videoTrack.getConstraints();

constraints.frameRate = { max: 20, min: 20 };

setTimeout(() => {
videoTrack.applyConstraints(constraints);

}, 5000);

})();
```

[Radar WebKit Bug Importer]

<rdar://problem/80868463>

[youenn fablet]

Thanks for the report Manik, I think this is covered by https://bugs.webkit.org/show_bug.cgi?id=228634.

[youenn fablet]

*** This bug has been marked as a duplicate of bug 228634 ***

[Manik]

Thanks Youenn - appreciate the fix!

[Manik]

@Youenn,

We reviewed the patch for the fix and it looks like it will only help if the new FPS constraints is the same as originally requested. I believe it does not fix our issue where we request an FPS os 30 (or default) and use applyConstraints to request 20, 15, etc thereafter.

I'm reopening the issue, please let me know what you think.

[youenn fablet]

Well, we do change the camera setup to match the application frame rate.
The alternative would be to always go to high frame rate and do software frame rate decimation.

To prevent this optimization, you can use track clone to keep a track with high frame rate and another track with a lower frame rate. This should work fine with https://bugs.webkit.org/show_bug.cgi?id=228634 fix.

For instance: https://jsfiddle.net/6qLsm7jp/ (doTest1 should recalibrate, doTest2 should not).
Can you clarify whether you would like doTest1 to not recalibrate either?