WebRTC replaceTrack adding latency

[Marc]

I'm using the following code to switch cameras on iOS. It works, however it seems to add around 1s of latency to the video. Each time I switch cameras it appears to grow. I have managed to replicate it in Safari Technology Preview.

navigator.mediaDevices
  .getUserMedia({
    video: {
      deviceId: {
        exact: selectedCamera
      }
    }
  })
  .then(function(stream) {
    let videoTrack = stream.getVideoTracks()[0];
      var sender = pc.getSenders().find(function(s) {
        return s.track.kind == videoTrack.kind;
      });
      sender.replaceTrack(videoTrack);
  })
  .catch(function(err) {
    console.error('Error happens:', err);
  });

[Radar WebKit Bug Importer]

<rdar://problem/62924912>

[youenn fablet]

Hi Marc, just tried your code in Web Inspector on https://webrtc.github.io/samples/src/content/peerconnection/pc1/ and I do not see the added latency. Would you be able to provide a full example?

[Marc]

Sure. I'm testing this on my iPhone XS.

1. Load https://webrtc.github.io/samples/src/content/peerconnection/pc1/
2. Click Start (Allow) and then Call
3. Wave at the camera to show that its almost in sync
4. paste this code into the console:

let cameraIds = [];

navigator.mediaDevices.enumerateDevices().then(function(deviceInfos)
{
	for (let i = 0; i !== deviceInfos.length; ++i)
	{
		const deviceInfo = deviceInfos[i];
		if (deviceInfo.kind === 'videoinput')
		{
			cameraIds.push(deviceInfo.deviceId);
		}
	}
});
let idCount = 1;

function swapCamera()
{
	navigator.mediaDevices
	  .getUserMedia({
	    video: {
	      deviceId: {
	        exact: cameraIds[idCount]
	      }
	    }
	  })
	  .then(function(stream) {
		  console.log('gotstream');
		  localVideo.srcObject = stream;
	    let videoTrack = stream.getVideoTracks()[0];
	      var sender = pc1.getSenders().find(function(s) {
	        return s.track.kind == videoTrack.kind;
	      });
	      sender.replaceTrack(videoTrack);
	  })
	  .catch(function(err) {
	    console.error('Error happens:', err);
	  });
	  
	  idCount++;
	  if(idCount >= cameraIds.length)
	  {
		  idCount = 0;
	  }
 }

5. call swapCamera() function
6. call swapCamera() function again to return to first camera
7. wave at camera to see the delay.

[Marc]

Created attachment 398816 [details]
iOS replaceTrack Delay Example

Example video showing the latency. Notice the latency increases the more I switch cameras.

I'm unable to replicate this now on iPhone X running iOS 13.6. Could you retest Marc on iOS 13.6 and see if you still see the same thing?

[Marc]

(In reply to daginge from comment #5)
> I'm unable to replicate this now on iPhone X running iOS 13.6. Could you
> retest Marc on iOS 13.6 and see if you still see the same thing?

Thanks for looking into this. I think you are right. I can't seem to replicate it now.