Bug 180680 – Allow AudioContext to start when getUserMedia is on

• Component: WebRTC
• Status: RESOLVED FIXED

[youenn fablet]

Allow AudioContext to start when getUserMedia is on

[youenn fablet]

Created attachment 329057 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 329057 [details]
Patch

Clearing flags on attachment: 329057

Committed r225785: <https://trac.webkit.org/changeset/225785>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/35995701>

[Ben]

With this fix we could autoplay a video element that has audio as long as the user previously approved a getUserMedia request.

In current Safari stable on desktop it's not enough to approve a getUserMedia request in the beginning of the session. The user has to actively capture mic or webcam to make video autoplay work.
This a regression. In our app we have one user broadcasting and several viewers. Capturing their mic just to make autoplay work doesn't make sense.

[youenn fablet]

> In current Safari stable on desktop it's not enough to approve a
> getUserMedia request in the beginning of the session. The user has to
> actively capture mic or webcam to make video autoplay work.
> This a regression. In our app we have one user broadcasting and several
> viewers. Capturing their mic just to make autoplay work doesn't make sense.

The principle is that a user should make a gesture to activate sound.
It can be the getUserMedia prompt, it can also be a click on a video element, play button, "activate sound" button.
Once a page is producing audio content, other video elements should autoplay.

I am not sure what your exact request is and what the regression you are pointing at is.

[Ben]

I'm creating AudioContext and playing it as a response of user gesture. I can hear the noise from the AudioContext but later when I'm trying to autoplay a video element with audio it is muted unless I'm actively capturing the local mic/cam. 

This is the callback of click event to enable audio:

enableAudio() {
  let audioContext = 'AudioContext' in window ? new AudioContext() : new window.webkitAudioContext();
  // create 2 seconds buffer
  let buffer = audioContext.createBuffer(2, audioContext.sampleRate*2, audioContext.sampleRate);
  
  // create noise
  for (var channel = 0; channel < 2; channel++) {
    // This gives us the actual ArrayBuffer that contains the data
    var nowBuffering = buffer.getChannelData(channel);
    for (var i = 0; i < audioContext.sampleRate*2; i++) {
      // Math.random() is in [0; 1.0]
      // audio needs to be in [-1.0; 1.0]
      nowBuffering[i] = Math.random() * 2 - 1;
    }
  }

  let source = audioContext.createBufferSource();
  source.buffer = buffer;
  source.connect(audioContext.destination);
  source.start(0);

  // create a PeerConnection and try to autoplay remote video+audio
}

[youenn fablet]

To summarize, your issue is:
- AudioContext is started on user click and produces audio
- video element is being added later on and will not autoplay even though web audio is producing audio

There are two workarounds I can think of right now:
- play the audio of the video element through AudioContext instead of video elements
- When AudioContext is being clicked, call play() on the video element

[Ben]

Thank you for the workaround. Mixing all the audio and playing with a single AudioContext work with WebRTC streams but I think will break lip sync.
It also doesn't help with autoplay HLS and YouTube videos on the web conference.