Bug 210285 - [iOS] MediaStream can only be used ~10 times on new video elements
Summary: [iOS] MediaStream can only be used ~10 times on new video elements
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Safari 13
Hardware: iPhone / iPad iOS 13
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-04-09 09:48 PDT by Dustin Greif
Modified: 2020-09-16 01:57 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dustin Greif 2020-04-09 09:48:35 PDT
We have a web app that uses `getUserMedia` to grab a feed from the users camera and capture images from it.  As the user navigates the app, the camera feed is not always visible (and thus removed from the dom).  To improve efficiency, we make a single call to `getUserMedia` and hold onto the returned MediaStream and then bind that stream to a new video element using `srcObject` each time the user is ready to capture new images.  This works great in all browsers except safari on iOS.  What we see on iOS is that stream can be played ~10-13 times in new video elements, and after that the `loadedmetadata` event will never fire on the video element.  We can make another call to `getUserMedia` at that point to get a new stream, but we cannot find any indication that the video element, media stream, or video track have entered a bad state.  Calling `play()` on the video element simply does not resolve or throw an error.  If we use the exact same video element each time, the issue doesn't seem to happen as quickly.  What we would expect is that the video stream can be bound to an infinite number of video elements, as long as the previous element has been removed from the dom.  Here is a simple recreation of the problem:

<html>
  <body>
    <div id="output">
      <h1 id="count">Please allow camera permissions, then the example will start...</h1>
    </div>

    <script>
      function start() {
        const output = document.getElementById('output'),
          countHeader = document.getElementById('count'),
          shareVideoElement = false

        let count = 0
        let video, stream

        if (shareVideoElement) {
          video = document.createElement('video')
        }

        function show() {
          if (!shareVideoElement) {
            video = document.createElement('video')
          }

          video.srcObject = stream
          video.style.border = '1px solid black'

          output.appendChild(video)
          count++
          countHeader.innerHTML = count

          const timer = setTimeout(() => {
            console.log(`FAILED to play stream after ${count} attempts`)
            countHeader.innerHTML += ' - Failure detected'
            console.log(video, stream, stream.getVideoTracks()[0])
          }, 1000)

          video.play().then(() => {
            clearTimeout(timer)
            setTimeout(hide, 100)
          }).catch(e => {
            alert(e)
            console.log(e)
          })
        }

        function hide() {
          if (count >= 50) {
            countHeader.innerHTML = 'Successfully reached 50 attempts'
            return
          }

          if (video) {
            video.parentNode.removeChild(video)
          }

          setTimeout(show, 10)
        }

        function loadNewStream() {
          navigator.mediaDevices.getUserMedia({ video: true }).then(s => {
            stream = s
            show()
          }).catch(e => {
            console.error(e)
            countHeader.innerHTML = 'Failed to access camera: ' + e
          })
        }

        loadNewStream()
      }

      start()
    </script>
  </body>
</html>
Comment 1 Radar WebKit Bug Importer 2020-04-09 15:50:27 PDT
<rdar://problem/61546009>
Comment 2 youenn fablet 2020-04-10 07:20:15 PDT
Thanks for the report, I also reproduced it.
I changed it a bit in https://jsfiddle.net/8r0czdus/

This does not repro for mock video capture source.
This probably does not repro for RTCPeerConnection remote video tracks either.

From logging, it seems that video capture stops without WebKit getting any notification except than not receiving any video frame.
Capture sometimes restarts, according logging, it seems that some video elements are GCed. So it seems there is a conflict of resources.

Destroying the underlying player seems to fix it, which probably goes well with a resource conflict issue.

As a temporary workaround, the video element srcObject attribute could be set to null when moved out of the DOM.
If I change the fiddle to do that (https://jsfiddle.net/ud7w64hy/) this is working for me.
Comment 3 youenn fablet 2020-04-10 07:20:24 PDT
Cannot repro on MacOS
Comment 4 Dustin Greif 2020-04-10 08:14:02 PDT
Thanks for the quick response!  I just tried setting srcObject to null before the video element is removed from the DOM and it worked great!  Hopefully this can get a permanent fix, but for now that work around should work well for our use case.  Thank you!