Bug 199661 – Allow MediaStream and non MediaStream backed videos to play together

• Component: WebRTC
• Status: RESOLVED FIXED

[Ben]

This commit says:
"Any non MediaStream backed video will stop all MediaStream backed video elements."
https://trac.webkit.org/changeset/245712/webkit

In our app the meeting organizer share mp4 or youtube video. The video will stop WebRTC streams and make the app broken.
Why is it important to prevent webrtc streams to play with non MediaStream videos?

[Ben]

Hi youenn,

Any workaround that will let us play a MediaStream backed video with a non MediaStream video at the same time?

I think we are already affected by this on iOS and can't play a non MediaStream backed video (HLS) when there is AudioContext playing a mix of serveral MediaStream audio.

[Miguel]

Hi Youenn,

about this update: https://trac.webkit.org/changeset/245712/webkit/

Our app is a modern webinar platform that has a feature to play a HLS video to share it with your audience together with multiple WebRTC videos backed with MediaStreams.

The issue that exists only in Safari is that the WebRTC videos backed with MediaStreams are paused when a non MediaStreams backed video is played. This is clearly intentional as per the description in the changelog:

Any non-MediaStream backed video will stop all MediaStream backed video elements.
Conversely, all non MediaStream backed videos will stop when playing one MediaStream backed video.

The use case for playing non MediaStreams backed videos together with MediaStreams backed videos is important for Webinar or Video-call platforms. 

Please, consider to solve it to standardize this behavior with other browsers. Meanwhile, what would be the work-around to make this feature work in Safari? 

We have ended up using a pair of <audio> and <video muted> tags to be able to play a non MediaStreams backed video together with MediaStream backed videos. This has removed the synchronization between audio and video tracks. We have not considered using a media controller because it status is not clear or documented.   

Thank you all!

[Radar WebKit Bug Importer]

<rdar://problem/68622411>

[youenn fablet]

Created attachment 408906 [details]
Patch

[Miguel]

Thanks for the quick patch Youenn! Is there any way to assign it to a reviewer to move it forward?

[Jer Noble]

Comment on attachment 408906 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=408906&action=review

> Source/WebCore/platform/audio/PlatformMediaSession.cpp:356
> -    return m_client.hasMediaStreamSource() && otherSession.m_client.hasMediaStreamSource();
> +    return m_client.hasMediaStreamSource() || otherSession.m_client.hasMediaStreamSource();

I don't think this is correct; we shouldn't allow a Media Stream in one document context to play simultaneously to a YouTube video in an entirely separate document context.  Can we narrow this to enable playback only when the two sessions are both in the same document?

[youenn fablet]

(In reply to Jer Noble from comment #6)
> Comment on attachment 408906 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=408906&action=review
> 
> > Source/WebCore/platform/audio/PlatformMediaSession.cpp:356
> > -    return m_client.hasMediaStreamSource() && otherSession.m_client.hasMediaStreamSource();
> > +    return m_client.hasMediaStreamSource() || otherSession.m_client.hasMediaStreamSource();
> 
> I don't think this is correct; we shouldn't allow a Media Stream in one
> document context to play simultaneously to a YouTube video in an entirely
> separate document context.  Can we narrow this to enable playback only when
> the two sessions are both in the same document?

True, that is an existing issue, let's fix it in https://bugs.webkit.org/show_bug.cgi?id=220504.

[youenn fablet]

Comment on attachment 408906 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=408906&action=review

>>> Source/WebCore/platform/audio/PlatformMediaSession.cpp:356
>>> +    return m_client.hasMediaStreamSource() || otherSession.m_client.hasMediaStreamSource();
>> 
>> I don't think this is correct; we shouldn't allow a Media Stream in one document context to play simultaneously to a YouTube video in an entirely separate document context.  Can we narrow this to enable playback only when the two sessions are both in the same document?
> 
> True, that is an existing issue, let's fix it in https://bugs.webkit.org/show_bug.cgi?id=220504.

Looking at PlatformMediaSessionManager::sessionWillBeginPlayback, it seems that we will not even interrupt sessions if they do not have the same media session type.
Meaning that audio would not interrupt VideoAudio for instance. Not sure this is expected either.

[youenn fablet]

Created attachment 417961 [details]
Rebasing

[EWS]

commit-queue failed to commit attachment 417961 [details] to WebKit repository.

[EWS]

Committed r271698: <https://trac.webkit.org/changeset/271698>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 417961 [details].