Summary: | getUserMedia framerate unusable under low light in iOS 12.2 | ||||||
---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | daginge | ||||
Component: | WebRTC | Assignee: | youenn fablet <youennf> | ||||
Status: | RESOLVED FIXED | ||||||
Severity: | Major | CC: | commit-queue, eric.carlson, ggaren, ibc, jhenderson2177, jonlee, kevinday, michael, simon.fraser, webkit-bug-importer, youennf | ||||
Priority: | P2 | Keywords: | InRadar | ||||
Version: | Safari 12 | ||||||
Hardware: | iPhone / iPad | ||||||
OS: | Unspecified | ||||||
Attachments: |
|
Description
daginge
2019-03-25 14:07:46 PDT
Able to replicate this on my iPad Pro as well. Software version: 16E227 Model name: iPad Pro 10.5 inch Model number: MQDW2KN/A (A1701) Interestingly, the problem fixes itself (as does the resolution (??)) if I go to the home screen and open Safari again. To repro: 1. Go to https://webrtc.github.io/samples/src/content/getusermedia/gum/ 2. Click "open camera" 3. Observe that frame rate is slow 4. Go to the home screen by swiping up 5.Wait at least 2 seconds 6. Open Safari again 7. Observe that the resolution has changed (black borders around the video frame), and the video is now fast again (as fast as native camera app) Problem also persists with all standard resolutions tried here: https://webrtc.github.io/samples/src/content/getusermedia/resolution/. Workaround by going to the home screen and back seems to fix the issue (and changes the aspect ration) for all resolutions. I've seen this issue in Macbook Pro OSX latest versions and Safari Tech latest versions. "Sometimes" it does not happen. No idea what it depends on, but most times it happens. As of this morning I'm unable to replicate the issue as seen in the videos in either of my devices. None of the devices have been shut off during the night. https://photos.app.goo.gl/4hmARduvZd9jrFND9 Aspect ratio still changes when going to the home screen and back though, but at least the horrible lag isn't there anymore. I FINALLY FIGURED IT OUT! iOS 12.2 behaves badly in _low light_ scenarios. That's why it didn't work in the evening with poor lighting conditions, and worked well in daylight when I tested it again. https://photos.app.goo.gl/akjHTfQ6oAZtfwCW8 See attached video showing extremely poor performance in low light, then going to home screen and back magically fixes the problem (and changes aspect ratio). Moving to a better lit scenario also fixes the problem. This was not a problem in previous versions. Do I get a cake now? :D > Do I get a cake now? :D
Thanks!
I was also able to repro on iOS and somehow on MacOS as well.
The call to setActiveVideoMinFrameDuration/setActiveVideoMaxFrameDuration in AVVideoCaptureSource::setSizeAndFrameRateWithPreset might actually trigger this behavior. I'm able to reproduce this on my iPhone X (MQCN2LL/A A1865) running iOS 12.2 (16E227) if I don't have perfect lighting. Another workaround is to request "frameRate: 60" in the constraints and the problem goes away, regardless of the lighting quality. Lower frameRate values, or not specifying one, and the problem still happens. Created attachment 369305 [details]
Patch
Comment on attachment 369305 [details]
Patch
r=me
Comment on attachment 369305 [details] Patch Clearing flags on attachment: 369305 Committed r245032: <https://trac.webkit.org/changeset/245032> All reviewed patches have been landed. Closing bug. Comment on attachment 369305 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369305&action=review > Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm:332 > + if (requestedFrameRate < frameRateRange.minFrameRate) > + requestedFrameRate = frameRateRange.minFrameRate; > + else if (requestedFrameRate > frameRateRange.maxFrameRate) > + requestedFrameRate = frameRateRange.maxFrameRate; Use clampTo()? *** Bug 197703 has been marked as a duplicate of this bug. *** (In reply to Simon Fraser (smfr) from comment #15) > Comment on attachment 369305 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=369305&action=review > > > Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm:332 > > + if (requestedFrameRate < frameRateRange.minFrameRate) > > + requestedFrameRate = frameRateRange.minFrameRate; > > + else if (requestedFrameRate > frameRateRange.maxFrameRate) > > + requestedFrameRate = frameRateRange.maxFrameRate; > > Use clampTo()? Makes sense, will fix it in bug 197704. (In reply to JH from comment #10) > I'm able to reproduce this on my iPhone X (MQCN2LL/A A1865) running iOS 12.2 > (16E227) if I don't have perfect lighting. > > Another workaround is to request "frameRate: 60" in the constraints and the > problem goes away, regardless of the lighting quality. Lower frameRate > values, or not specifying one, and the problem still happens. Although the frame rate problem seems to go away with this workaround, the black bars seem to still be an issue. I wonder if the patches fix the black bars too? (In reply to micsun-al from comment #18) > (In reply to JH from comment #10) > > I'm able to reproduce this on my iPhone X (MQCN2LL/A A1865) running iOS 12.2 > > (16E227) if I don't have perfect lighting. > > > > Another workaround is to request "frameRate: 60" in the constraints and the > > problem goes away, regardless of the lighting quality. Lower frameRate > > values, or not specifying one, and the problem still happens. > > Although the frame rate problem seems to go away with this workaround, the > black bars seem to still be an issue. I wonder if the patches fix the black > bars too? That is a separate issue, I filed https://bugs.webkit.org/show_bug.cgi?id=197707. This issue is not fixed for me on iOS 12.3.1 using Safari 12.1.1. I can still reliably reproduce this in low light on https://webrtc.github.io/samples/src/content/getusermedia/gum/ on my iPhone X. Let me know if you need additional debugging information. See difference between native camera and webrtc: native: https://share.icloud.com/photos/0oKCLxUqdxlPfFSjxDg7eGwPQ webrtc: https://share.icloud.com/photos/0hYZ4piriFK45trA0hk_gPJ9g These were recorded right after one another in the exact same lighting conditions. 12.3.1 doesn't have the fix for this bug. Fixing a bug in bugzilla does not correspond with when it's available in an OS update. |