Bug 199661 - Allow MediaStream and non MediaStream backed videos to play together
Summary: Allow MediaStream and non MediaStream backed videos to play together
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on: 220504
Blocks:
  Show dependency treegraph
 
Reported: 2019-07-09 23:43 PDT by Ben
Modified: 2021-01-21 10:09 PST (History)
10 users (show)

See Also:


Attachments
Patch (9.33 KB, patch)
2020-09-16 02:06 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Rebasing (9.35 KB, patch)
2021-01-20 05:30 PST, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ben 2019-07-09 23:43:36 PDT
This commit says:
"Any non MediaStream backed video will stop all MediaStream backed video elements."
https://trac.webkit.org/changeset/245712/webkit

In our app the meeting organizer share mp4 or youtube video. The video will stop WebRTC streams and make the app broken.
Why is it important to prevent webrtc streams to play with non MediaStream videos?
Comment 1 Ben 2019-07-29 13:46:44 PDT
Hi youenn,

Any workaround that will let us play a MediaStream backed video with a non MediaStream video at the same time?

I think we are already affected by this on iOS and can't play a non MediaStream backed video (HLS) when there is AudioContext playing a mix of serveral MediaStream audio.
Comment 2 Miguel 2020-09-09 22:37:45 PDT
Hi Youenn,

about this update: https://trac.webkit.org/changeset/245712/webkit/

Our app is a modern webinar platform that has a feature to play a HLS video to share it with your audience together with multiple WebRTC videos backed with MediaStreams.

The issue that exists only in Safari is that the WebRTC videos backed with MediaStreams are paused when a non MediaStreams backed video is played. This is clearly intentional as per the description in the changelog:

Any non-MediaStream backed video will stop all MediaStream backed video elements.
Conversely, all non MediaStream backed videos will stop when playing one MediaStream backed video.

The use case for playing non MediaStreams backed videos together with MediaStreams backed videos is important for Webinar or Video-call platforms. 

Please, consider to solve it to standardize this behavior with other browsers. Meanwhile, what would be the work-around to make this feature work in Safari? 

We have ended up using a pair of <audio> and <video muted> tags to be able to play a non MediaStreams backed video together with MediaStream backed videos. This has removed the synchronization between audio and video tracks. We have not considered using a media controller because it status is not clear or documented.   

Thank you all!
Comment 3 Radar WebKit Bug Importer 2020-09-09 23:21:13 PDT
<rdar://problem/68622411>
Comment 4 youenn fablet 2020-09-16 02:06:13 PDT
Created attachment 408906 [details]
Patch
Comment 5 Miguel 2021-01-08 02:21:59 PST
Thanks for the quick patch Youenn! Is there any way to assign it to a reviewer to move it forward?
Comment 6 Jer Noble 2021-01-08 16:17:07 PST
Comment on attachment 408906 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=408906&action=review

> Source/WebCore/platform/audio/PlatformMediaSession.cpp:356
> -    return m_client.hasMediaStreamSource() && otherSession.m_client.hasMediaStreamSource();
> +    return m_client.hasMediaStreamSource() || otherSession.m_client.hasMediaStreamSource();

I don't think this is correct; we shouldn't allow a Media Stream in one document context to play simultaneously to a YouTube video in an entirely separate document context.  Can we narrow this to enable playback only when the two sessions are both in the same document?
Comment 7 youenn fablet 2021-01-11 03:25:58 PST
(In reply to Jer Noble from comment #6)
> Comment on attachment 408906 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=408906&action=review
> 
> > Source/WebCore/platform/audio/PlatformMediaSession.cpp:356
> > -    return m_client.hasMediaStreamSource() && otherSession.m_client.hasMediaStreamSource();
> > +    return m_client.hasMediaStreamSource() || otherSession.m_client.hasMediaStreamSource();
> 
> I don't think this is correct; we shouldn't allow a Media Stream in one
> document context to play simultaneously to a YouTube video in an entirely
> separate document context.  Can we narrow this to enable playback only when
> the two sessions are both in the same document?

True, that is an existing issue, let's fix it in https://bugs.webkit.org/show_bug.cgi?id=220504.
Comment 8 youenn fablet 2021-01-11 03:29:02 PST
Comment on attachment 408906 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=408906&action=review

>>> Source/WebCore/platform/audio/PlatformMediaSession.cpp:356
>>> +    return m_client.hasMediaStreamSource() || otherSession.m_client.hasMediaStreamSource();
>> 
>> I don't think this is correct; we shouldn't allow a Media Stream in one document context to play simultaneously to a YouTube video in an entirely separate document context.  Can we narrow this to enable playback only when the two sessions are both in the same document?
> 
> True, that is an existing issue, let's fix it in https://bugs.webkit.org/show_bug.cgi?id=220504.

Looking at PlatformMediaSessionManager::sessionWillBeginPlayback, it seems that we will not even interrupt sessions if they do not have the same media session type.
Meaning that audio would not interrupt VideoAudio for instance. Not sure this is expected either.
Comment 9 youenn fablet 2021-01-20 05:30:34 PST
Created attachment 417961 [details]
Rebasing
Comment 10 EWS 2021-01-21 09:28:38 PST
commit-queue failed to commit attachment 417961 [details] to WebKit repository.
Comment 11 EWS 2021-01-21 10:09:00 PST
Committed r271698: <https://trac.webkit.org/changeset/271698>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 417961 [details].