Bug 176439 – Safari on iOS11 Freezes when viewing a cropped remote video

• Status: RESOLVED
• Resolution: FIXED
• Component: WebRTC

[Adam]

Steps to reproduce:

1. Visit https://output.jsbin.com/pihucav in Chrome on a Mac and allow access to your camera
2. Visit https://output.jsbin.com/pihucav in Safari on iOS and allow access to your camera

Result: After a few seconds (the timing seems to change) your whole iPhone/iPad will freeze. You have to force restart it.

A few other observations:

1. This does not happen with this sample app: https://jsbin.com/mizarag which is identical except that it is using the fitMode 'contain' on the Video Element. It also doesn't happen if the Video is sized to the right aspect ratio. It seems to have something to do with cropping the video.
2. It doesn't happen if the other party is using Safari, either on iOS or Mac. It does happen if the other party is Chrome or Firefox though.
3. This does not happen if the iPhone or iPad is in Landscape mode. It only seems to happen in Portrait.

[Adam]

It also happens if the video is too wide to fit inside the browser window.

[Radar WebKit Bug Importer]

<rdar://problem/34281364>

[youenn fablet]

https://output.jsbin.com/pihucav does not work for me now, I'll retry later on.
Is it Safari that is freezing or the whole OS?

[Adam]

(In reply to youenn fablet from comment #3)
> https://output.jsbin.com/pihucav does not work for me now, I'll retry later
> on.
> Is it Safari that is freezing or the whole OS?

pihucav is working for me, I just checked again. This also happens with meet.tokbox.com.

The whole OS is freezing, not just Safari.

[Adam]

Here is a stripped down WebRTC example (without using opentok.js) that reproduces the problem https://output.jsbin.com/vopimuw

1. Open the URL using Google Chrome on a Mac
2. Open it in Safari on iOS

Result: iOS freezes up completely.

[Adam]

Actually, I want to modify the steps a little bit.

1. Visit https://output.jsbin.com/vopimuw in Google Chrome on a Mac and allow access to the camera/mic.
2. Visit https://output.jsbin.com/vopimuw in Safari on iOS. 
3. Wait for the remote stream to load before you allow access to camera/mic.
4. Once the remote stream has loaded click "Allow".
5. Scroll down to view your own stream.

[Adam]

Similarly if I set object-fit:contain on the video element I don't get the freeze. 

https://jsbin.com/nogaxe/edit?html,css,js,output

Also see https://bugs.webkit.org/show_bug.cgi?id=175014
Aka rdar://problem/33781374

This is possibly a dupe of that, but I'm not sure. That freeze has been known and reproducible for 3-4 betas before the final iOS 11 release, but still wasn't addressed in time for the final release.

Cross your fingers that it gets fixed soon, I guess.

[Ben]

We get whole OS freeze in iPhone running iOS 11.02. Need hard reboot.
object-fit:contain doesn't help.
Adam, can  you suggest any other workaround?

[youenn fablet]

Ben, can you provide a test web site.
It might also be related to https://bugs.webkit.org/show_bug.cgi?id=175014

[Mike Block]

I had opened another bug with the same issue:

https://bugs.webkit.org/show_bug.cgi?id=178357

If I ensure that I contain the full video in the outer container, then it seems to prevent freezing of the OS.

Latest test on 11.0.3 - no resolution yet.

[Mike Block]

Note that the local stream continues to feed to the remote peers after the screen locks up. Audio also seems to continue flowing both directions (need more testing on this). If I wait a few minutes, sometimes the device releases and starts working again, but the page needs a reload to restart.

[Ben]

Mike, can you please explain what do you mean by this?
| contain the full video in the outer container

[Mike Block]

It seems that the problem is managed by ensuring that none of the video is cropped within the container it resides in. For example, if I set a DIV with a 4:3 aspect ratio on the page and place a VIDEO tag inside this div, so long as the complete boundaries of the video tag do not overflow the containing DIV, it seems to be okay.

Where it reliably freezes is when I allow the video tag to be 100% of the width of the video and auto height, and use overflow: hidden in CSS to hide the parts of the video that are extending top and bottom outside the container.

Does that make sense?

[Ben]

Thanks. I think I understand but not sure how to handle it in a real app where you can't always control the aspect ratio of the other peer.
I've tried to remove overflow:hidden, width:100% and height:100% from several containers but it doesn't help.

[Mike Block]

@Ben the video should letterbox itself if you constrain the video tag. Try a setup like this for your test:

<style>
/*
fixed size for each video wrapper
*/
.video-wrap {
     width: 320px;
     height: 240px;
     position: relative;
}
/*
video ELEMENT expands to the dimensions of the wrapper
actual video will be left right letter boxed and smaller if portrait
it will fill if landscape and 4:3 source
if 16:9 source, it will be top-bottom letter boxed
*/
.video-wrap > video {
     position: absolute;
     top: 0; left: 0;
     width: 100%;
     max-width: 100%;
     height: 100%;
     max-height: 100%;
}
</style>
<!-- add as many of these as you need and attach streams as needed -->
<div class="video-wrap">
     <video muted autoplay playsinline>
</div>

[Ben]

I gave video-wrap size 320x240 and even 160x120. I see the video element in a letterbox. The iPhone still freezes when I rotate it from portrait to landscape several times.
I might still have some problematic style or there is an additional issue.

[Mike Block]

The issue may then be with re-rendering when switching camera orientations. Not sure what to suggest.

[youenn fablet]

The issue might be more related to the actual video stream.
Rendering tricks might allow to reduce the risk of hitting this issue but will not probably work 100% of the time.

[Ben]

Is there something that can be changed in the getUserMedia constraints of the other peer or the SDP to workaround it?

[Alexey Proskuryakov]

*** Bug 178357 has been marked as a duplicate of this bug. ***

This is most likely a dupe of https://bugs.webkit.org/show_bug.cgi?id=175014

[youenn fablet]

Latest iOS 11.2 beta might solve the issue.
Let me known if this is still not working, otherwise I will close this bug in a few weeks.

[Adam]

Just tested with iOS 11.2 beta and the issue is resolved for me. Thank you so much!

[iamtesting]

iOS 11.2.5: Safari: Following does not work yet.

<video id="localVideo"  oncontextmenu="return false;"></video>
<video id="miniVideo"  oncontextmenu="return false;" autoplay="autoplay" muted="true"></video>

<script>
var mainVideoSeflView = document.getElementById('localVideo');//works
var pictureInPicture = document.getElementById('miniVideo');

// =============== Fail fail fail fail????? 
// (iOS iPad works, OSX Safari works, Google chrome works, Firefox works, Opera works


but not in iPhone???)

// =============== Fail fail fail fail????? 
pictureInPicture.srcObject = mainVideoSeflView.srcObject;
// FAIL FAIL FIAL

</script>