214689 2020-07-23 09:23:31 -0700 getUserMedia aspectRatio overconstrained in portrait orientation 2020-07-24 09:30:01 -0700 1 1 1 Unclassified WebKit WebRTC Safari 13 iPhone / iPad iOS 13 NEW InRadar P2 Normal --- 1 kpd400 webkit-unassigned webkit-bug-importer youennf oldest_to_newest 1674249 0 405046 kpd400 2020-07-23 09:23:31 -0700 Created attachment 405046 Demo of aspect ratio bug in portrait mode I have a use case that requires me to get a 4:3 video from getUserMedia on iPad / iOS devices. I can do this reliably when the device is in landscape by setting the `aspectRatio` constraint to 4/3 but when the device is in portrait (thus requiring a relative 3/4 aspect ratio) an `OverconstrainedError: Invalid constraint` error is thrown. If I specify exact dimensions, e.g. 640x480 in landscape or 480x640 in portrait, I am able to obtain a 4:3 video regardless of orientation but this requires exact dimensions which is problematic when trying to obtain a maximum camera resolution that may or may not be supported by the device. I am using an iPad Pro (12.9-inch) (3rd generation) on iPadOS 13.6 and I have attached a demo to reproduce the issue when tested on a mobile device in portrait mode. 1674459 1 webkit-bug-importer 2020-07-23 16:34:59 -0700 <rdar://problem/66020174> 1674628 2 youennf 2020-07-24 07:23:33 -0700 I confirmed the issue. 1674632 3 youennf 2020-07-24 07:34:13 -0700 In general, I would recommend to use ideal constraints for getUserMedia. Then, you can check the results and use applyConstraints to resize at will. Would the following workaround work for you: - Call getUserMedia with aspectRatio 4/3 as ideal constraints whatever the orientation. - Use applyConstraints to flip height and width in case of portrait mode. 1674634 4 youennf 2020-07-24 07:39:53 -0700 In case we can resize frames at will, we could also potentially have arbitrary aspect ratio as well. 1674664 5 405149 kpd400 2020-07-24 09:03:15 -0700 Created attachment 405149 applyConstraint Workaround Test (In reply to youenn fablet from comment #3) > In general, I would recommend to use ideal constraints for getUserMedia. > Then, you can check the results and use applyConstraints to resize at will. > > Would the following workaround work for you: > - Call getUserMedia with aspectRatio 4/3 as ideal constraints whatever the > orientation. > - Use applyConstraints to flip height and width in case of portrait mode. Thanks for the quick response/workaround. I'm seeing odd behavior when initially applying constraints in portrait mode - I have to flip the height/width with applyConstraints multiple times before it actually crops the video. This does not seem to be an issue in landscape. The dimensions set with applyConstraints do get reflected in getSettings, but only reliably crop the video after "flipping" 3-5 times. However, once the video is reliably cropping it works exactly as expected and would be an acceptable workaround. A new demo with this workaround is attached. 1674675 6 youennf 2020-07-24 09:18:22 -0700 https://bugs.webkit.org/show_bug.cgi?id=214741 should allow to set an aspect ratio of 3/4. Let's keep this bug open until we get a good idea of how we want to handle rotation. 1674677 7 youennf 2020-07-24 09:19:34 -0700 (In reply to youenn fablet from comment #6) > https://bugs.webkit.org/show_bug.cgi?id=214741 should allow to set an aspect > ratio of 3/4. > Let's keep this bug open until we get a good idea of how we want to handle > rotation. And fix the potential bug of needing to call applyConstraints multiple times. 1674680 8 kpd400 2020-07-24 09:22:35 -0700 (In reply to youenn fablet from comment #7) > (In reply to youenn fablet from comment #6) > > https://bugs.webkit.org/show_bug.cgi?id=214741 should allow to set an aspect > > ratio of 3/4. > > Let's keep this bug open until we get a good idea of how we want to handle > > rotation. > > And fix the potential bug of needing to call applyConstraints multiple times. Awesome, thanks Youenn! 1674681 9 youennf 2020-07-24 09:23:18 -0700 > crop the video after "flipping" 3-5 times. However, once the video is Doing your test, it seems to work for me after clicking 2 times. 1674692 10 kpd400 2020-07-24 09:30:01 -0700 (In reply to youenn fablet from comment #9) > > crop the video after "flipping" 3-5 times. However, once the video is > > Doing your test, it seems to work for me after clicking 2 times. I'm seeing the following in portrait: Click 1 - No change Click 2 - Crops to 4:3 Click 3 - No change Click 4 - No change Click 5+ - Always crops, when in 3:4 aspect ratio the video is "zoomed" compared to the original 3:4 video. 405046 2020-07-23 09:23:31 -0700 2020-07-23 09:23:31 -0700 Demo of aspect ratio bug in portrait mode aspect_ratio_repro.html text/html 1978 kpd400 PCFET0NUWVBFIGh0bWw+CjxodG1sPgoKPGhlYWQ+CiAgPG1ldGEgY2hhcnNldD0idXRmLTgiPgog IDx0aXRsZT5TYWZhcmkgQXNwZWN0UmF0aW8gQ29uc3RyYWludCBQb3J0cmFpdCBCdWc8L3RpdGxl Pgo8L2hlYWQ+Cgo8Ym9keT4KICA8ZGl2IGlkPSJ3cmFwcGVyIj4KICAgIDxkaXYgaWQ9ImNvbnRy b2xzIj4KICAgICAgPGJ1dHRvbiBpZD0iZXhhY3QtZGltZW5zaW9ucyI+UmVxdWVzdCBFeGFjdCBE aW1lbnNpb25zPC9idXR0b24+CiAgICAgIDxidXR0b24gaWQ9ImFzcGVjdC1yYXRpbyI+UmVxdWVz dCBBc3BlY3QgUmF0aW88L2J1dHRvbj4KICAgIDwvZGl2PgogICAgPHAgaWQ9ImVycm9yIj48L3A+ CiAgICA8ZGl2IGlkPSJjb250YWluZXIiPgogICAgICA8dmlkZW8gaWQ9InZpZGVvIiBwbGF5c2lu bGluZT0idHJ1ZSIgYXV0b3BsYXk9InRydWUiPjwvY2FudmFzPgogICAgPC9kaXY+CiAgPC9kaXY+ CgogIDxzY3JpcHQ+CiAgICBjb25zdCB2aWRlbyA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCd2 aWRlbycpCiAgICBjb25zdCBlcnJvciA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdlcnJvcicp CgogICAgY29uc3QgbGFuZHNjYXBlRXhhY3REaW1lbnNpb25zID0gewogICAgICB3aWR0aDogeyBl eGFjdDogNjQwIH0sCiAgICAgIGhlaWdodDogeyBleGFjdDogNDgwIH0KICAgIH0sCiAgICAgIHBv cnRyYWl0RXhhY3REaW1lbnNpb25zID0gewogICAgICAgIHdpZHRoOiBsYW5kc2NhcGVFeGFjdERp bWVuc2lvbnMuaGVpZ2h0LAogICAgICAgIGhlaWdodDogbGFuZHNjYXBlRXhhY3REaW1lbnNpb25z LndpZHRoCiAgICAgIH0sCiAgICAgIGxhbmRzY2FwZUFzcGVjdFJhdGlvID0gewogICAgICAgIGFz cGVjdFJhdGlvOiB7IGV4YWN0OiA0IC8gMyB9CiAgICAgIH0sCiAgICAgIHBvcnRyYWl0QXNwZWN0 UmF0aW8gPSB7CiAgICAgICAgYXNwZWN0UmF0aW86IHsgZXhhY3Q6IDMgLyA0IH0KICAgICAgfSwK ICAgICAgaXNQb3J0cmFpdCA9ICgpID0+IG1hdGNoTWVkaWEoJyhvcmllbnRhdGlvbjogcG9ydHJh aXQpJykubWF0Y2hlcwoKICAgIGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdleGFjdC1kaW1lbnNp b25zJykuYWRkRXZlbnRMaXN0ZW5lcignY2xpY2snLCBhc3luYyAoKSA9PiB7CiAgICAgIHZpZGVv LnNyY09iamVjdCA9IG51bGwKICAgICAgY29uc3Qgc3RyZWFtID0gYXdhaXQgbmF2aWdhdG9yLm1l ZGlhRGV2aWNlcy5nZXRVc2VyTWVkaWEoewogICAgICAgIHZpZGVvOiBpc1BvcnRyYWl0KCkgPyBw b3J0cmFpdEV4YWN0RGltZW5zaW9ucyA6IGxhbmRzY2FwZUV4YWN0RGltZW5zaW9ucwogICAgICB9 KS5jYXRjaChlID0+IHsKICAgICAgICBlcnJvci5pbm5lclRleHQgPSBlLm5hbWUgKyAnOiAnICsg ZS5tZXNzYWdlCiAgICAgICAgdGhyb3cgZQogICAgICB9KQoKICAgICAgaWYgKHN0cmVhbSkgewog ICAgICAgIGVycm9yLmlubmVyVGV4dCA9ICcnCiAgICAgICAgdmlkZW8uc3JjT2JqZWN0ID0gc3Ry ZWFtCiAgICAgIH0KICAgIH0pCgogICAgZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ2FzcGVjdC1y YXRpbycpLmFkZEV2ZW50TGlzdGVuZXIoJ2NsaWNrJywgYXN5bmMgKCkgPT4gewogICAgICB2aWRl by5zcmNPYmplY3QgPSBudWxsCiAgICAgIGNvbnN0IHN0cmVhbSA9IGF3YWl0IG5hdmlnYXRvci5t ZWRpYURldmljZXMuZ2V0VXNlck1lZGlhKHsKICAgICAgICB2aWRlbzogaXNQb3J0cmFpdCgpID8g cG9ydHJhaXRBc3BlY3RSYXRpbyA6IGxhbmRzY2FwZUFzcGVjdFJhdGlvCiAgICAgIH0pLmNhdGNo KGUgPT4gewogICAgICAgIGVycm9yLmlubmVyVGV4dCA9IGUubmFtZSArICc6ICcgKyBlLm1lc3Nh Z2UKICAgICAgICB0aHJvdyBlCiAgICAgIH0pCgogICAgICBpZiAoc3RyZWFtKSB7CiAgICAgICAg ZXJyb3IuaW5uZXJUZXh0ID0gJycKICAgICAgICB2aWRlby5zcmNPYmplY3QgPSBzdHJlYW0KICAg ICAgfQogICAgfSkKICA8L3NjcmlwdD4KPC9ib2R5PgoKPC9odG1sPg== 405149 2020-07-24 09:03:15 -0700 2020-07-24 09:03:15 -0700 applyConstraint Workaround Test apply_constraints_workaround.html text/html 2201 kpd400 PCFET0NUWVBFIGh0bWw+CjxodG1sPgoKPGhlYWQ+CiAgPG1ldGEgY2hhcnNldD0idXRmLTgiPgog IDx0aXRsZT5TYWZhcmkgQXNwZWN0UmF0aW8gQXBwbHkgQ29uc3RyYWludCBQb3J0cmFpdCBXb3Jr YXJvdW5kPC90aXRsZT4KPC9oZWFkPgoKPGJvZHk+CiAgPGRpdiBpZD0id3JhcHBlciI+CiAgICA8 ZGl2IGlkPSJjb250cm9scyI+CiAgICAgIDxidXR0b24gaWQ9InRvZ2dsZS12aWRlbyI+U3RhcnQg NDozIFZpZGVvPC9idXR0b24+CiAgICAgIDxidXR0b24gaWQ9ImZsaXAtdmlkZW8iIGRpc2FibGVk PSJ0cnVlIj5BcHBseSBGbGlwcGVkIERpbWVuc2lvbnM8L2J1dHRvbj4KICAgIDwvZGl2PgogICAg PHAgaWQ9Im1lc3NhZ2UiPjwvcD4KICAgIDxkaXYgaWQ9ImNvbnRhaW5lciI+CiAgICAgIDx2aWRl byBpZD0idmlkZW8iIHBsYXlzaW5saW5lPSJ0cnVlIiBhdXRvcGxheT0idHJ1ZSI+PC9jYW52YXM+ CiAgICA8L2Rpdj4KICA8L2Rpdj4KCiAgPHNjcmlwdD4KICAgIGNvbnN0IHZpZGVvID0gZG9jdW1l bnQuZ2V0RWxlbWVudEJ5SWQoJ3ZpZGVvJykKICAgIGNvbnN0IG1lc3NhZ2UgPSBkb2N1bWVudC5n ZXRFbGVtZW50QnlJZCgnbWVzc2FnZScpCiAgICBjb25zdCB0b2dnbGVCdXR0b24gPSBkb2N1bWVu dC5nZXRFbGVtZW50QnlJZCgndG9nZ2xlLXZpZGVvJykKICAgIGNvbnN0IGZsaXBCdXR0b24gPSBk b2N1bWVudC5nZXRFbGVtZW50QnlJZCgnZmxpcC12aWRlbycpCiAgICBjb25zdCBpc1BvcnRyYWl0 ID0gKCkgPT4gbWF0Y2hNZWRpYSgnKG9yaWVudGF0aW9uOiBwb3J0cmFpdCknKS5tYXRjaGVzCgog ICAgdmFyIG1lZGlhU3RyZWFtID0gdW5kZWZpbmVkCiAgICBkb2N1bWVudC5nZXRFbGVtZW50QnlJ ZCgndG9nZ2xlLXZpZGVvJykuYWRkRXZlbnRMaXN0ZW5lcignY2xpY2snLCBhc3luYyAoKSA9PiB7 CiAgICAgIGlmIChtZWRpYVN0cmVhbSkgewogICAgICAgIHZpZGVvLnNyY09iamVjdCA9IG51bGwK ICAgICAgICBtZWRpYVN0cmVhbSA9IHVuZGVmaW5lZAogICAgICAgIG1lc3NhZ2UuaW5uZXJUZXh0 ID0gJycKICAgICAgICBmbGlwQnV0dG9uLmRpc2FibGVkID0gdHJ1ZQogICAgICAgIHRvZ2dsZUJ1 dHRvbi5pbm5lclRleHQgPSAnU3RhcnQgNDozIFZpZGVvJwogICAgICB9IGVsc2UgewogICAgICAg IGNvbnN0IHN0cmVhbSA9IGF3YWl0IG5hdmlnYXRvci5tZWRpYURldmljZXMuZ2V0VXNlck1lZGlh KHsKICAgICAgICAgIHZpZGVvOiB7IGFzcGVjdFJhdGlvOiB7IGlkZWFsOiA0IC8gMyB9IH0KICAg ICAgICB9KS5jYXRjaChlID0+IHsKICAgICAgICAgIG1lc3NhZ2UuaW5uZXJUZXh0ID0gZS5uYW1l ICsgJzogJyArIGUubWVzc2FnZQogICAgICAgICAgdGhyb3cgZQogICAgICAgIH0pCgogICAgICAg IGlmIChzdHJlYW0pIHsKICAgICAgICAgIG1lZGlhU3RyZWFtID0gc3RyZWFtCiAgICAgICAgICB2 aWRlby5zcmNPYmplY3QgPSBzdHJlYW0KICAgICAgICAgIGZsaXBCdXR0b24uZGlzYWJsZWQgPSBm YWxzZQogICAgICAgICAgdG9nZ2xlQnV0dG9uLmlubmVyVGV4dCA9ICdTdG9wIFZpZGVvJwogICAg ICAgIH0KICAgICAgfQogICAgfSkKCiAgICBmbGlwQnV0dG9uLmFkZEV2ZW50TGlzdGVuZXIoJ2Ns aWNrJywgYXN5bmMgKCkgPT4gewogICAgICBpZiAoIW1lZGlhU3RyZWFtKSB7CiAgICAgICAgcmV0 dXJuCiAgICAgIH0KICAgICAgY29uc3QgW3ZpZGVvVHJhY2tdID0gbWVkaWFTdHJlYW0uZ2V0Vmlk ZW9UcmFja3MoKQogICAgICBjb25zdCBvbGRTZXR0aW5ncyA9IHZpZGVvVHJhY2suZ2V0U2V0dGlu Z3MoKQogICAgICBjb25zdCBjb25zdHJhaW50cyA9IHsKICAgICAgICB3aWR0aDogb2xkU2V0dGlu Z3MuaGVpZ2h0LAogICAgICAgIGhlaWdodDogb2xkU2V0dGluZ3Mud2lkdGgKICAgICAgfQogICAg ICBhd2FpdCB2aWRlb1RyYWNrLmFwcGx5Q29uc3RyYWludHMoY29uc3RyYWludHMpCiAgICAgIGNv bnN0IG5ld1NldHRpbmdzID0gdmlkZW9UcmFjay5nZXRTZXR0aW5ncygpCiAgICAgIG1lc3NhZ2Uu aW5uZXJUZXh0ID0gYEJlZm9yZTogJHtvbGRTZXR0aW5ncy53aWR0aH14JHtvbGRTZXR0aW5ncy5o ZWlnaHR9IHwgQXBwbGllZDogJHtjb25zdHJhaW50cy53aWR0aH14JHtjb25zdHJhaW50cy5oZWln aHR9IHwgQWZ0ZXI6ICR7bmV3U2V0dGluZ3Mud2lkdGh9eCR7bmV3U2V0dGluZ3MuaGVpZ2h0fWAK ICAgIH0pCiAgPC9zY3JpcHQ+CjwvYm9keT4KCjwvaHRtbD4=