RESOLVED CONFIGURATION CHANGED Bug 179363
iOS calling getUserMedia() again kills video display of first getUserMedia() 
https://bugs.webkit.org/show_bug.cgi?id=179363
Summary iOS calling getUserMedia() again kills video display of first getUserMedia()
Chad Phillips
Reported 2017-11-06 23:16:54 PST
Verified by the reference code below: On iOS, a second call to getUserMedia() kills the display of a video stream obtained by an earlier call to getUserMedia(). The original stream displays fine until the subsequent getUserMedia() call, then goes black. Note that this doesn't happen on Desktop Safari, only on iOS Safari in my tests. Reference code: <!DOCTYPE html> <html> <body> <div> <video id="video1" autoplay playsinline></video> </div> <script type="text/javascript"> var constraints1 = { audio: false, video: { height: { max: 480, }, width: { max: 640, }, }, }; navigator.mediaDevices.getUserMedia(constraints1).then(function(stream) { var video1 = document.getElementById('video1'); video1.srcObject = stream; }).catch(function(err) { console.error("Device access checks failed: ", err, constraints1); }); var constraints2 = { audio: false, video: true, }; navigator.mediaDevices.getUserMedia(constraints2).then(function(stream) { }).catch(function(err) { console.error("Device access checks failed: ", err, constraints2); }); </script> </body> </html>
Attachments
Multiple getUserMedia() streams controlled by UI. (3.41 KB, text/html)
2017-11-07 14:58 PST, Chad Phillips 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Chad Phillips
Comment 1 2017-11-07 14:58:03 PST
Created attachment 326267 [details] Multiple getUserMedia() streams controlled by UI. Attaching a more robust test case, allowing user triggering of multiple streams obtained by getUserMedia(). Both 'Video 1' and 'Video 2' can be started and stopped fine as long as the other is not actively streaming. But if you start one while another is streaming to screen, the first stream goes black.
Chad Phillips
Comment 2 2017-11-22 20:05:49 PST
I've spent some more time digging into this issue, and it turns out that the video MediaStreamTrack element of video 1 has its 'mute' property set to true upon another gUM call that requests a video stream. It's not even necessary for this gUM call to do anything with the video stream (like display it) for the muting of the previous video MediaStreamTrack element. Furthermore, I see no way via the API to unmute the muted video track -- the 'mute' property is read-only, and toggling the 'enabled' property of either video track has no effect on its state. Is this issue related to the 'Multiple Simultaneous Audio or Video Streams' as noted at https://developer.apple.com/library/content/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html ? If so, it's going to be severely limiting for certain multiparty videoconferencing applications. For example: - It's common practice to show a user their own local video feed with one (higher resolution) stream, and publish another (lower resolution) stream to other users - To accommodate receivers with different bandwidth capabilities, a common practice is to publish both a high resolution and a low resolution stream
Chad Phillips
Comment 3 2018-04-23 21:14:27 PDT
This is still an issue with iOS 11.3, would love somebody to have a look at it. IMO, it's unnecessarily limiting to block video after a user has granted access to the camera, as mentioned in the specific use cases in previous comments. Limitations like this make it so that the in-browser WebRTC experience on iOS is the worst of any platform -- is that really what Apple wants??
Dag-Inge Aas
Comment 4 2018-06-28 01:16:35 PDT
Just chiming in here. This is a blocker for us in switching between front and back camera on Safari iOS. At least with minimal disruption to the user experience.
youenn fablet
Comment 5 2018-06-29 11:01:08 PDT
> - It's common practice to show a user their own local video feed with one > (higher resolution) stream, and publish another (lower resolution) stream to > other users Understood that this is not optimal, although in most UI, the local video track usually takes a smaller part of the screen than the remote video track. Also applyConstraints should be the solution for changing the resolution of a given video track. At this point, it is not feasible to have two tracks with different resolutions. Ideally, this should be feasible using MediaStream cloning and applyConstraints. Note that general support for multiple video streams might not always be feasible, in particular if the streams are coming from multiple capture devices. > - To accommodate receivers with different bandwidth capabilities, a common > practice is to publish both a high resolution and a low resolution stream Simulcast might be a better option there. (In reply to daginge from comment #4) > Just chiming in here. This is a blocker for us in switching between front > and back camera on Safari iOS. At least with minimal disruption to the user > experience. I would be interested in what disruption you encounter. The following is expected to work without too much trouble: navigator.mediaDevices.getUserMedia({video: { facingMode: "user"} }).then((s) => { localVideo.srcObject = s; peerConnection.getSenders()[0].replaceTrack(s.getVideoTracks()[0]); });
Chad Phillips
Comment 6 2018-06-30 17:23:24 PDT
> Understood that this is not optimal, although in most UI, > the local video track usually takes a smaller part of the > screen than the remote video track. It's not the only rational UI choice though. I have a layout where all feeds, including the local user's feed, are the same size, and many users prefer this layout. Also, it doesn't seem like a very flexible architectural mindset to make that kind of assumption about how a designer wants to lay things out? > Note that general support for multiple video streams might not > always be feasible, in particular if the streams are coming from > multiple capture devices. I want to point out here that Chrome on Android has zero restrictions/limitations in this regard. You can call gUM multiple times, grab different resolutions, clone streams, etc., and it all works flawlessly. By comparison, iOS is a nightmare to work on for anything beyond the most basic use cases. Which also makes the end user experience worse on iOS because of all the compromises necessary. It's puzzling to me Apple implemented WebRTC at all if they're going to hamstring it. > Simulcast might be a better option there. Not all clients support simulcast. For example, Chrome doesn't yet support it for h264, which is of course the required codec if you want interop with iOS devices.
Chad Phillips
Comment 7 2018-08-31 10:24:29 PDT
Adding some further clarification from more testing: 1. This issue only occurs when a subsequent gUM() request asks for an already requested media type. For example, if gUM() #1 asks for video, and gUM() #2 also asks for video, gUM() #1's video stream is affected. However, if gUM() #2 only asks for audio, then gUM() #1's video stream is NOT affected. 2. Because the mechanism is setting the track's muted property, data is still sent along a peer connection, although it's not very useful since the other side only receives muted video.
Chad Phillips
Comment 8 2018-08-31 10:39:21 PDT
This issue also occurs for audio tracks. I now believe the issue can be fully summarized as: If a getUserMedia() requests a media type requested in a previous getUserMedia(), the previously requested media track's 'muted' property is set to true, and there is no way to programmatically unmute it.
youenn fablet
Comment 9 2018-08-31 10:57:37 PDT
This is currently an expected behavior. There seems to be two requests: - Allow multiple capture using the same capture device with different parameters (resolution, frameRate...). - Allow capture on two different capture devices at the same time.
Chad Phillips
Comment 10 2018-08-31 11:35:00 PDT
@youenn, those seem correct, and are supported by every other platform I've tried.
Radar WebKit Bug Importer
Comment 11 2018-08-31 12:11:49 PDT
<rdar://problem/43950488>
Ramya D
Comment 12 2018-12-18 21:20:31 PST
Whether it is fixed or not. I'm also facing the same issue @Chad Phillips
Ramya D
Comment 13 2018-12-19 03:46:11 PST
Please make this issue as a major one and why this is still not assigned.
youenn fablet
Comment 14 2018-12-19 08:27:19 PST
*** Bug 192849 has been marked as a duplicate of this bug. ***
Shreyas
Comment 15 2019-08-27 11:36:38 PDT
Are there any updates on this? I am working on an implementation where I need two videos in the same window. I tried MediaStream.clone() but it has its own issues. It would be great to have this feature.
youenn fablet
Comment 16 2019-11-18 00:51:32 PST
(In reply to Shreyas from comment #15) > Are there any updates on this? I am working on an implementation where I > need two videos in the same window. I tried MediaStream.clone() but it has > its own issues. It would be great to have this feature. What are the issues you encountered? cloning and applyConstraints should allow you to have two tracks with the same source and with different sizes.
Shreyas
Comment 17 2019-12-03 12:12:00 PST
I resolved this by another approach but here's what I was facing: With clone(), When I try to apply constraints on a new video, it also applies to the original video which did not fit my solution.
seba kerckhof
Comment 18 2020-07-09 02:19:47 PDT
Can we ever expect to see this fixed? Sure, we can usually modify our code to work around it, but it doesn't mean it's not another bug in an already very buggy webRTC implementation offered by webkit.
Jaya Allamsetty
Comment 19 2020-09-10 14:42:51 PDT
Are there any updates on this one ? Jitsi users are also facing this issue with both audio and video when switching camera because of how device selection UI is designed. We do a second gUM call for previewing the tracks.
Maximilian Böhm
Comment 20 2020-12-08 00:32:30 PST
So just to be sure, it is currently not supported to "Allow capture on two different capture devices at the same time", right? I suppose it is not planned to change this behaviour as this report is on "NEW" since 2017? We would have a valid use case: One should be seen while he or she is presenting physical stuff. Think of a car show, you should see the one who is trying to sell as well as the car/interior/exterior. This is a highly asked feature, especially during covid...
Olivier Anguenot
Comment 21 2021-06-30 23:14:42 PDT
Had the same problem with audio only tracks. In my use case, I create several PeerConnections to connect to several audio rooms in FreeSwitch using Janus as a WebRTC Gateway. I faced the same issue: my first line was muted as soon as the second was created. Additionally to that, the complexity was that the WebRTC part is mainly managed by Janus and the SIP plugin... But hopefully, there is a way to access to the PeerConnection object from Janus. The solution I found is a workaround and fixing that issue will for sure help to handle the case correctly. 1/ I added an handler on the "onmute" event on the local track associated to each room connected. 2/ When one of my local track goes to "muted", I try to find in the other existing local tracks, a track which is not muted 3/ I clone that track 4/ I use the PeerConnection -> Sender -> replaceTrack() function "Magically", it seems to work. I'm able to speak and be heard in the room associated to that track. But, not sure on what is the consequence on that (listeners for example...)
Trace
Comment 22 2021-08-13 04:58:33 PDT
13-08-2021: Facing the same issue with Jitsi. Hope it will be fixed soon!
awe.media
Comment 23 2021-08-24 23:26:41 PDT
BUMP! Adding streams from multiple cameras (e.g. 'environment' and 'user' facingMode) would be very useful and works fine in Chrome on Android. This bug is nearly 4 years old now and still marked as NEW 8(
Austin
Comment 24 2021-09-29 15:16:56 PDT
Currently seeing this still in my company's application. Any updates on getting a fix?
sak126p
Comment 25 2022-02-09 02:44:49 PST
any news? problem still occurs on safari 15.3
Brent Fulgham
Comment 26 2022-02-12 21:48:30 PST
This is actually: <rdar://42467754>
Mitch Talmadge
Comment 27 2022-02-15 14:23:31 PST
Adding my voice to this. This is really frustrating for our iOS users in our meeting software. Please dedicate time to get this fixed.
Mitch Talmadge
Comment 28 2022-02-16 08:59:38 PST
I've also found that using the SpeechRecognition API (https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition) will cause the microphone track to become muted everywhere else, including in WebRTC calls. Due to this bug, it is impossible to generate captions using this API in our meeting software without making the user go silent for everyone else. We had to disable caption generation for this reason.
Eugene M. Joseph
Comment 29 2022-03-27 16:15:57 PDT
Seconding everything here. Unable to locally capture an audio WebRTC stream via the MediaRecorder on iOS. The originally WebRTC fails to work when the MediaRecorder instance is started. Would be very useful to work with simultaneous media streams.
youenn fablet
Comment 30 2022-03-28 07:14:54 PDT
Bug 237359 fixes the case of muting a video track when another video track is created on iOS in https://commits.webkit.org/r291034. The new behavior in iOS is that, if a new camera device is used, we will stop the previous capture and start the new one as the current API we are using does not allow to capture both at the same time. If the new capture is using the same camera as the previous one, both will be able to continue in parallel. If your particular use case is about capturing with both cameras, please file a new bug.
youenn fablet
Comment 31 2022-03-28 07:16:27 PDT
(In reply to Austin from comment #24) > Currently seeing this still in my company's application. Any updates on > getting a fix? The current workaround until the above fix rolls out is to clone the track instead of calling getUserMedia. This allows to get the same video feed in two tracks, and potentially apply width/height/frame rate constraints independently.
awe.media
Comment 32 2022-03-29 00:26:08 PDT
(In reply to youenn fablet from comment #30) > Bug 237359 fixes the case of muting a video track when another video track > is created on iOS in https://commits.webkit.org/r291034. > > The new behavior in iOS is that, if a new camera device is used, we will > stop the previous capture and start the new one as the current API we are > using does not allow to capture both at the same time. If the new capture is > using the same camera as the previous one, both will be able to continue in > parallel. > > If your particular use case is about capturing with both cameras, please > file a new bug. In case this wasn't automatically linked back - see https://bugs.webkit.org/show_bug.cgi?id=238492 Thanks.
Sean
Comment 33 2022-11-07 16:20:47 PST
> If the new capture is using the same camera as the previous one, both will be able to continue in parallel. I still get a black screen (after 3 secs) on the second call to getUserMedia() on iOS in this example where the same device is returned: <!DOCTYPE html> <html> <body> <div> <video id="video1" autoplay playsinline></video> </div> <script type="text/javascript"> (async function () { var constraints = { audio: false, video: true, }; const result1 = await navigator.mediaDevices.getUserMedia(constraints).then(async function(stream) { var video1 = document.getElementById('video1'); video1.srcObject = stream; return stream; }); console.log(result1.getTracks()[0].getCapabilities().deviceId); window.setTimeout(async function () { const result2 = await navigator.mediaDevices.getUserMedia(constraints); console.log(result2.getTracks()[0].getCapabilities().deviceId); }, 3000); }()); </script> </body> </html> Is this expected?
Note You need to log in before you can comment on or make changes to this bug.