Bug 214689 – getUserMedia aspectRatio overconstrained in portrait orientation

• Component: WebRTC
• Status: NEW

[Kyle Dunn]

Created attachment 405046 [details]
Demo of aspect ratio bug in portrait mode

I have a use case that requires me to get a 4:3 video from getUserMedia on iPad / iOS devices. I can do this reliably when the device is in landscape by setting the `aspectRatio` constraint to 4/3 but when the device is in portrait (thus requiring a relative 3/4 aspect ratio) an `OverconstrainedError: Invalid constraint` error is thrown.

If I specify exact dimensions, e.g. 640x480 in landscape or 480x640 in portrait, I am able to obtain a 4:3 video regardless of orientation but this requires exact dimensions which is problematic when trying to obtain a maximum camera resolution that may or may not be supported by the device.

I am using an iPad Pro (12.9-inch) (3rd generation) on iPadOS 13.6 and I have attached a demo to reproduce the issue when tested on a mobile device in portrait mode.

[Radar WebKit Bug Importer]

<rdar://problem/66020174>

[youenn fablet]

I confirmed the issue.

[youenn fablet]

In general, I would recommend to use ideal constraints for getUserMedia.
Then, you can check the results and use applyConstraints to resize at will.

Would the following workaround work for you:
- Call getUserMedia with aspectRatio 4/3 as ideal constraints whatever the orientation.
- Use applyConstraints to flip height and width in case of portrait mode.

[youenn fablet]

In case we can resize frames at will, we could also potentially have arbitrary aspect ratio as well.

[Kyle Dunn]

Created attachment 405149 [details]
applyConstraint Workaround Test

(In reply to youenn fablet from comment #3)
> In general, I would recommend to use ideal constraints for getUserMedia.
> Then, you can check the results and use applyConstraints to resize at will.
> 
> Would the following workaround work for you:
> - Call getUserMedia with aspectRatio 4/3 as ideal constraints whatever the
> orientation.
> - Use applyConstraints to flip height and width in case of portrait mode.

Thanks for the quick response/workaround. I'm seeing odd behavior when initially applying constraints in portrait mode - I have to flip the height/width with applyConstraints multiple times before it actually crops the video. This does not seem to be an issue in landscape. The dimensions set with applyConstraints do get reflected in getSettings, but only reliably crop the video after "flipping" 3-5 times. However, once the video is reliably cropping it works exactly as expected and would be an acceptable workaround. 

A new demo with this workaround is attached.

[youenn fablet]

https://bugs.webkit.org/show_bug.cgi?id=214741 should allow to set an aspect ratio of 3/4.
Let's keep this bug open until we get a good idea of how we want to handle rotation.

[youenn fablet]

(In reply to youenn fablet from comment #6)
> https://bugs.webkit.org/show_bug.cgi?id=214741 should allow to set an aspect
> ratio of 3/4.
> Let's keep this bug open until we get a good idea of how we want to handle
> rotation.

And fix the potential bug of needing to call applyConstraints multiple times.

[Kyle Dunn]

(In reply to youenn fablet from comment #7)
> (In reply to youenn fablet from comment #6)
> > https://bugs.webkit.org/show_bug.cgi?id=214741 should allow to set an aspect
> > ratio of 3/4.
> > Let's keep this bug open until we get a good idea of how we want to handle
> > rotation.
> 
> And fix the potential bug of needing to call applyConstraints multiple times.

Awesome, thanks Youenn!

[youenn fablet]

> crop the video after "flipping" 3-5 times. However, once the video is

Doing your test, it seems to work for me after clicking 2 times.

[Kyle Dunn]

(In reply to youenn fablet from comment #9)
> > crop the video after "flipping" 3-5 times. However, once the video is
> 
> Doing your test, it seems to work for me after clicking 2 times.

I'm seeing the following in portrait:

Click 1  - No change
Click 2  - Crops to 4:3
Click 3  - No change
Click 4  - No change
Click 5+ - Always crops, when in 3:4 aspect ratio the video is "zoomed" compared to the original 3:4 video.