Bug 186889 – WebRTC MediaStreamTrack Enable / Disable causes video delay / lag

• Component: WebRTC
• Status: RESOLVED FIXED

[Neil Kinnish]

This is a similar issue reported and fixed on chrome:

https://bugs.chromium.org/p/webrtc/issues/detail?id=4355

Enable / Disable a WebRTC MediaStreamTrack and the video is delayed and suffers lag, refreshing fixes so it definitely seems to be caused by this event. Disable / enable is used to mute the audio or video and is necessary.

[Neil Kinnish]

To add more context... the video will be affected even if you are only changing an audio track.

[Neil Kinnish]

In further tests, even without Video, just audio. Disabling and enabling causes Safari to lag remote streams and eventually fail. I'll try and get more information on the eventual fail by debugging a bit more.

[Neil Kinnish]

I don't have the full info, but it appears the stream ends completely when you set enable to false, this is more noticeable when only video or audio as having two and one remaining enabled would keep the stream alive. 

Hope this helps.

[Neil Kinnish]

To add... this seems intermittent, but easily reproducible if you just enable/disable (mute audio for example).

[Neil Kinnish]

Further testing with both video + audio.

Video tracks set to enabled false cause less issues than audio set to enabled false. There is also a noticeable delay changing audio compared to video. Audio is also what seems to cause lag in video I haven't seen this when video tracks enabled/disabled.

[Radar WebKit Bug Importer]

<rdar://problem/41370285>

[youenn fablet]

Hi Neil, thanks for the report.
Can you clarify whether the issue is with capture tracks or remote tracks backed by a peer connection?
If you have a repel case that you could share,that would be great.

[Radar WebKit Bug Importer]

<rdar://problem/41370427>

[Neil Kinnish]

Hi

> Can you clarify whether the issue is with capture tracks or remote tracks backed by a peer connection?

Capture tracks, I'm not seeing the issues on remote tracks, at least I haven't seen anything as bad.

> If you have a repel case that you could share,that would be great.

I don't have anything I can share right now I'm afraid. If I get time I'll try and put something together but not sure when I will have time.

I think my report is essentially 2 issues (they may be connected):

1. The issue where the video track becomes delayed when you disable audio (lag) does seem to very similar to the issue chrome use to experience as per that link.

2. The intermittent stream fails (mainly when you only have video or audio – not both), don't get an error but it seems as nothing is being sent.

As per last comment it's really noticeable that disabling audio tracks is slow compared to video.

Thanks for responding.

[Neil Kinnish]

Okay, I've done a lot more testing.

1. I restarted the IOS device which I was seeing the stream fails on and also got hold of another 2 IOS devices. I don't seem to be able to re-create that particular issue now, so could have just been a temp issue on the device!? 

2. The original issue – video delay after muting (disabling) audio track however is still present. The video being sent is noticeably delayed on the other end after muting.

I can't send you my project, but to test this case I just downloaded: https://github.com/shanet/WebRTC-Example

I added an audio toggle...

```
function toggleAudio() {
  if (localStream.getAudioTracks()[0].enabled) {
    localStream.getAudioTracks()[0].enabled = false;
  } else {
    localStream.getAudioTracks()[0].enabled = true;
  }
}
```

It causes the same issue, the more you mute the more delay.

Hope this helps, sorry for so many pings.

[Neil Kinnish]

PS... I tested on IOS 11 and macOS 10.13.3, happened on both.

[youenn fablet]

(In reply to Neil Kinnish from comment #11)
> PS... I tested on IOS 11 and macOS 10.13.3, happened on both.

Thanks Neil for all this information, this will help us figure out the issue.

[Neil Kinnish]

Just to add, in further tests I've noticed that overtime the video will lag generally... this is exaggerated when the stream is enabled / disabled. Mainly tested on iOS.

[youenn fablet]

Created attachment 344438 [details]
Patch

[youenn fablet]

Created attachment 344441 [details]
Patch

[youenn fablet]

Created attachment 344444 [details]
Patch

[Eric Carlson]

Comment on attachment 344444 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=344444&action=review

> Source/ThirdParty/libwebrtc/ChangeLog:4
> +        https://bugs.webkit.org/show_bug.cgi?id=186889

Isn’t this the wrong bug title for this fix?

[youenn fablet]

Created attachment 344470 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 344470 [details]
Patch for landing

Clearing flags on attachment: 344470

Committed r233604: <https://trac.webkit.org/changeset/233604>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.