IOS 12.2 WebRTC Safari send track event with invalid MediaStream

• Component: WebRTC
• Status: RESOLVED CONFIGURATION CHANGED

[Xin]

Prior to IOS 12.2:
If for a 2 way call between IOS Safari and Chrome
for peer connection events particularly track events:

Safari will send: (happpens in order)
1. audio track event with parameters: audioTrack, mediaStreamA (mediaStreamA contains audioTrack)
2. video track event with parameters: videoTrack, mediaStreamA (mediaStreamA contains both audioTrack and videoTrack)
The 2 tracks shared the exact same container mediaStreamA


But now with IOS 12.2 Safari:
1. audio track event with parameters: audioTrack, mediaStreamA (mediaStreamA contains audioTrack)
2. video track event with parameters: videoTrack, mediaStreamB (mediaStreamB contains only videoTrack)
The 2 tracks are located in the different mediaStream object. (different id, and each only contains its own track)

Please investigate this, as I can see, this breaks quite a lot different products.

Thanks.



Xin Wang

[Radar WebKit Bug Importer]

<rdar://problem/49462060>

[youenn fablet]

Hi Xin,
Do you have a repro case?
I am wondering in particular whether addTrack is called with no second parameter (stream). If you set the stream parameter, does it work for you?

[Xin]

(In reply to youenn fablet from comment #2)
> Hi Xin,
> Do you have a repro case?
> I am wondering in particular whether addTrack is called with no second
> parameter (stream). If you set the stream parameter, does it work for you?

Hi Youenn,

I have manually tested as below

If chrome-to-chrome:
chrome pc call pc.addTrack(track, stream1, stream2)
when event fired on the remote chrome, the event (RTCTrackEvent), `event.streams` contains 2 streams.

If safari-to-chrome:
safari pc call pc.addTrack(track, stream1, stream2)
when event fired on the remote chrome, the event (RTCTrackEvent), `event.streams` contains 1 stream.

It may only support single stream, but with 12.2, the stream is not the original one (https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/addTrack)

[Xin]

Hi Youenn,

Any further help you need from me? How is your side going? It can be reproduced every time on the 12.2.

Let me know what extra information you need from me.

[youenn fablet]

(In reply to Xin from comment #4)
> Hi Youenn,
> 
> Any further help you need from me? How is your side going? It can be
> reproduced every time on the 12.2.
> 
> Let me know what extra information you need from me.

I am still not clear on the request here.
The initial issue is that two streams are created.

I would believe that calling addTrack(audioTrack, stream) and addTrack(videoTrack, stream) will allow to get on the other side a single stream with two tracks.
Can you confirm that and whether that fixes your issue?

Your second message seems to indicate that only one stream is used by Safari, probably to generate the SDP so that Chrome then only creates one stream.
That seems like a different bug to me.

Can you clarify your issue?

[Xin]

(In reply to youenn fablet from comment #5)
> (In reply to Xin from comment #4)
> > Hi Youenn,
> > 
> > Any further help you need from me? How is your side going? It can be
> > reproduced every time on the 12.2.
> > 
> > Let me know what extra information you need from me.
> 
> I am still not clear on the request here.
> The initial issue is that two streams are created.
> 
> I would believe that calling addTrack(audioTrack, stream) and
> addTrack(videoTrack, stream) will allow to get on the other side a single
> stream with two tracks.
> Can you confirm that and whether that fixes your issue?
> 
> Your second message seems to indicate that only one stream is used by
> Safari, probably to generate the SDP so that Chrome then only creates one
> stream.
> That seems like a different bug to me.
> 
> Can you clarify your issue?


Hi Youenn,

Thank you.

Yes, it should be 2 issues, but it may be related to each other, and that's why I mentioned both same time.

The first (main) one which this thread is also focusing on is that:
when safari call addTrack(audioTrack, stream) and addTrack(videoTrack, stream)
the remote party receives 2 track events with (audioTrack, stream1) and (videoTrack, stream2) (stream1 and stream2 are separate objects, and stream1 only contains audioTrack, stream2 only contains videoTrack)

This is different from ios prior to 12.2, which as you mentioned, the remote party with receive the same mediaStream object, which is the correct behaviour.

(ignore the second issue for this thread, I will check if there is some existing one, or need create new)

[Ryan Park]

I've been through same issue.
However, I couldn't find out the exact reason of the bugs.
I can only tell you how to reproduce the error.

Demonstrate the simplest video call through following website.
https://www.webrtc-experiment.com/video-conferencing

chrome-chrome
chrome-safari(12.1)
are fine.
But
safari(12.1)-safari(12.1) doesn't work.

Please run this simple demo and see what happen.

[youenn fablet]

I tried to repro the issue and came up with https://jsfiddle.net/z48159fw/1/
@Xin, can you try it?

@RyanPark, I am not sure what is not working for you in your provided example.
Note that one Safari page will mute capture of the other after a getUserMedia call.

[Xin]

Hi Youenn,

Thanks for the update, I have also tried that it should work on a single tab for the simulation (either chrome or safari, though it is a bit different), but if tries on different tabs (chrome-safari) the issue will appear.

https://jsfiddle.net/betimer/dm0k7832/

Can you please try the above? and use one chrome tab and one safari.
Follow the steps I have listed.
Go to the console, you can check the console warn logs.
You should be able to find the second track event, whose stream only contains single track data.

Thank you, Youenn.


Xin Wang

[youenn fablet]

> https://jsfiddle.net/betimer/dm0k7832/
> 
> Can you please try the above? and use one chrome tab and one safari.

I was able to repro with Safari 12.1 with Plan B.
This seems to affect answers only.
The only workaround I can think of right now is for the web application to munge the SDP to fix the faulty lines.

With Unified Plan enabled, the answer SDP seems to work correctly.

Can you try with the latest Safari Tech Preview?
Or with Safari 12.1 and unified plan enabled (Develop -> Experimental Features -> WebRTC Unified Plan).

We have seen increased reports of Safari not being able to receive audio as of late. Unfortunately I haven't been able to create proper tracking metrics for this bug yet before easter, so we only have customer support reports. 

We've tried to implement a workaround for this bug by creating our own MediaStream and doing addTrack on that instead of relying on the stream from the RTCTrackEvent. However, without a tracking metric I have no idea if this has solved the issue or not yet.

youenn, you mentioned some sdp munging tricks here, could you elaborate?

Trying to trick the bug by creating our own MediaStream and adding tracks didn't work.

youenn, could you elaborate on which lines are faulty in the sdp so we might work around this until unified plan drops?

[youenn fablet]

Unified plan should be enabled now and should fix that issue in latest iOS 12.3.
Let me know whether that works for you.