Bug 176439 - Safari on iOS11 Freezes when viewing a cropped remote video
Summary: Safari on iOS11 Freezes when viewing a cropped remote video
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Other
Hardware: iPhone / iPad iOS 11
: P2 Critical
Assignee: Nobody
URL:
Keywords: InRadar
: 178357 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-09-05 21:15 PDT by Adam
Modified: 2018-05-23 20:10 PDT (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adam 2017-09-05 21:15:40 PDT
Steps to reproduce:

1. Visit https://output.jsbin.com/pihucav in Chrome on a Mac and allow access to your camera
2. Visit https://output.jsbin.com/pihucav in Safari on iOS and allow access to your camera

Result: After a few seconds (the timing seems to change) your whole iPhone/iPad will freeze. You have to force restart it.

A few other observations:

1. This does not happen with this sample app: https://jsbin.com/mizarag which is identical except that it is using the fitMode 'contain' on the Video Element. It also doesn't happen if the Video is sized to the right aspect ratio. It seems to have something to do with cropping the video.
2. It doesn't happen if the other party is using Safari, either on iOS or Mac. It does happen if the other party is Chrome or Firefox though.
3. This does not happen if the iPhone or iPad is in Landscape mode. It only seems to happen in Portrait.
Comment 1 Adam 2017-09-05 21:33:42 PDT
It also happens if the video is too wide to fit inside the browser window.
Comment 2 Radar WebKit Bug Importer 2017-09-06 09:25:26 PDT
<rdar://problem/34281364>
Comment 3 youenn fablet 2017-09-06 09:31:31 PDT
https://output.jsbin.com/pihucav does not work for me now, I'll retry later on.
Is it Safari that is freezing or the whole OS?
Comment 4 Adam 2017-09-06 17:38:10 PDT
(In reply to youenn fablet from comment #3)
> https://output.jsbin.com/pihucav does not work for me now, I'll retry later
> on.
> Is it Safari that is freezing or the whole OS?

pihucav is working for me, I just checked again. This also happens with meet.tokbox.com.

The whole OS is freezing, not just Safari.
Comment 5 Adam 2017-09-07 23:16:06 PDT
Here is a stripped down WebRTC example (without using opentok.js) that reproduces the problem https://output.jsbin.com/vopimuw

1. Open the URL using Google Chrome on a Mac
2. Open it in Safari on iOS

Result: iOS freezes up completely.
Comment 6 Adam 2017-09-07 23:23:35 PDT
Actually, I want to modify the steps a little bit.

1. Visit https://output.jsbin.com/vopimuw in Google Chrome on a Mac and allow access to the camera/mic.
2. Visit https://output.jsbin.com/vopimuw in Safari on iOS. 
3. Wait for the remote stream to load before you allow access to camera/mic.
4. Once the remote stream has loaded click "Allow".
5. Scroll down to view your own stream.
Comment 7 Adam 2017-09-07 23:28:03 PDT
Similarly if I set object-fit:contain on the video element I don't get the freeze. 

https://jsbin.com/nogaxe/edit?html,css,js,output
Comment 8 andj2223 2017-09-21 00:29:56 PDT
Also see https://bugs.webkit.org/show_bug.cgi?id=175014
Aka rdar://problem/33781374

This is possibly a dupe of that, but I'm not sure. That freeze has been known and reproducible for 3-4 betas before the final iOS 11 release, but still wasn't addressed in time for the final release.

Cross your fingers that it gets fixed soon, I guess.
Comment 9 Ben 2017-10-07 06:54:05 PDT
We get whole OS freeze in iPhone running iOS 11.02. Need hard reboot.
object-fit:contain doesn't help.
Adam, can  you suggest any other workaround?
Comment 10 youenn fablet 2017-10-07 07:35:01 PDT
Ben, can you provide a test web site.
It might also be related to https://bugs.webkit.org/show_bug.cgi?id=175014
Comment 11 Mike Block 2017-10-18 10:33:36 PDT
I had opened another bug with the same issue:

https://bugs.webkit.org/show_bug.cgi?id=178357

If I ensure that I contain the full video in the outer container, then it seems to prevent freezing of the OS.

Latest test on 11.0.3 - no resolution yet.
Comment 12 Mike Block 2017-10-18 10:39:10 PDT
Note that the local stream continues to feed to the remote peers after the screen locks up. Audio also seems to continue flowing both directions (need more testing on this). If I wait a few minutes, sometimes the device releases and starts working again, but the page needs a reload to restart.
Comment 13 Ben 2017-10-18 14:34:00 PDT
Mike, can you please explain what do you mean by this?
| contain the full video in the outer container
Comment 14 Mike Block 2017-10-18 14:38:35 PDT
It seems that the problem is managed by ensuring that none of the video is cropped within the container it resides in. For example, if I set a DIV with a 4:3 aspect ratio on the page and place a VIDEO tag inside this div, so long as the complete boundaries of the video tag do not overflow the containing DIV, it seems to be okay.

Where it reliably freezes is when I allow the video tag to be 100% of the width of the video and auto height, and use overflow: hidden in CSS to hide the parts of the video that are extending top and bottom outside the container.

Does that make sense?
Comment 15 Ben 2017-10-18 15:20:10 PDT
Thanks. I think I understand but not sure how to handle it in a real app where you can't always control the aspect ratio of the other peer.
I've tried to remove overflow:hidden, width:100% and height:100% from several containers but it doesn't help.
Comment 16 Mike Block 2017-10-18 15:35:50 PDT
@Ben the video should letterbox itself if you constrain the video tag. Try a setup like this for your test:

<style>
/*
fixed size for each video wrapper
*/
.video-wrap {
     width: 320px;
     height: 240px;
     position: relative;
}
/*
video ELEMENT expands to the dimensions of the wrapper
actual video will be left right letter boxed and smaller if portrait
it will fill if landscape and 4:3 source
if 16:9 source, it will be top-bottom letter boxed
*/
.video-wrap > video {
     position: absolute;
     top: 0; left: 0;
     width: 100%;
     max-width: 100%;
     height: 100%;
     max-height: 100%;
}
</style>
<!-- add as many of these as you need and attach streams as needed -->
<div class="video-wrap">
     <video muted autoplay playsinline>
</div>
Comment 17 Ben 2017-10-18 16:06:45 PDT
I gave video-wrap size 320x240 and even 160x120. I see the video element in a letterbox. The iPhone still freezes when I rotate it from portrait to landscape several times.
I might still have some problematic style or there is an additional issue.
Comment 18 Mike Block 2017-10-18 16:19:24 PDT
The issue may then be with re-rendering when switching camera orientations. Not sure what to suggest.
Comment 19 youenn fablet 2017-10-18 16:21:57 PDT
The issue might be more related to the actual video stream.
Rendering tricks might allow to reduce the risk of hitting this issue but will not probably work 100% of the time.
Comment 20 Ben 2017-10-18 16:32:43 PDT
Is there something that can be changed in the getUserMedia constraints of the other peer or the SDP to workaround it?
Comment 21 Alexey Proskuryakov 2017-10-21 12:39:15 PDT
*** Bug 178357 has been marked as a duplicate of this bug. ***
Comment 22 andj2223 2017-10-26 16:28:54 PDT
This is most likely a dupe of https://bugs.webkit.org/show_bug.cgi?id=175014
Comment 23 youenn fablet 2017-11-01 16:47:40 PDT
Latest iOS 11.2 beta might solve the issue.
Let me known if this is still not working, otherwise I will close this bug in a few weeks.
Comment 24 Adam 2017-11-01 16:48:52 PDT
Just tested with iOS 11.2 beta and the issue is resolved for me. Thank you so much!
Comment 25 iamtesting 2017-12-20 13:17:27 PST
iOS 11.2.5: Safari: Following does not work yet.

<video id="localVideo"  oncontextmenu="return false;"></video>
<video id="miniVideo"  oncontextmenu="return false;" autoplay="autoplay" muted="true"></video>

<script>
var mainVideoSeflView = document.getElementById('localVideo');//works
var pictureInPicture = document.getElementById('miniVideo');

// =============== Fail fail fail fail????? 
// (iOS iPad works, OSX Safari works, Google chrome works, Firefox works, Opera works


but not in iPhone???)

// =============== Fail fail fail fail????? 
pictureInPicture.srcObject = mainVideoSeflView.srcObject;
// FAIL FAIL FIAL

</script>