180522 2017-12-06 22:41:06 -0800 Web audio without audio output should not require any user gesture on iOS 2022-05-06 13:26:09 -0700 1 1 1 Unclassified WebKit WebRTC Safari 11 iPhone / iPad iOS 11 NEW https://bugs.webkit.org/show_bug.cgi?id=180680 InRadar P2 Normal --- 1 adam webkit-unassigned andrew brad daginge jer.noble jonlee webkit-bug-importer youennf oldest_to_newest 1379039 0 adam 2017-12-06 22:41:06 -0800 We use the following code to analyse the audio input level in a local MediaStream coming from getUserMedia. This works fine in Safari on a Mac but on iOS you get constant 0 (because timeDomainData[idx] always returns 128). navigator.mediaDevices.getUserMedia({audio:true}).then(stream => { const context = new (window.AudioContext || window.webkitAudioContext)(); const sourceNode = context.createMediaStreamSource(stream); const analyser = context.createAnalyser(); sourceNode.connect(analyser); const timeDomainData = new Uint8Array(analyser.frequencyBinCount); setInterval(() => { analyser.getByteTimeDomainData(timeDomainData); let max = 0; for (let idx = 0; idx < timeDomainData.length; idx++) { max = Math.max(max, Math.abs(timeDomainData[idx] - 128)); } audioLevel.innerHTML = (max / 128); }, 100); }).catch(err => { alert(err.name + ' ' + err.message); }); You can see it working at https://output.jsbin.com/rexolan 1379042 1 adam 2017-12-06 22:49:15 -0800 It also does not work for remote streams. 1379152 2 youennf 2017-12-07 10:40:40 -0800 (In reply to Adam from comment #1) > It also does not work for remote streams. On iOS, AudioContext needs a user gesture. Can you retry by starting AudioContext as part of user gesture? We should probably remove that restriction when getUserMedia is on. 1379309 3 andrew 2017-12-07 15:12:14 -0800 > On iOS, AudioContext needs a user gesture. Is it intended that AudioContext needs a user gesture even when there is no audio output? Being able to visualize the audio volume would be especially useful in exactly this use case when audio output is blocked. 1379319 4 youennf 2017-12-07 15:23:03 -0800 (In reply to Andrew Morris from comment #3) > > On iOS, AudioContext needs a user gesture. > > Is it intended that AudioContext needs a user gesture even when there is no > audio output? Being able to visualize the audio volume would be especially > useful in exactly this use case when audio output is blocked. Good point, maybe the restriction should be targeted at the audio output. 1379639 5 webkit-bug-importer 2017-12-08 10:48:47 -0800 <rdar://problem/35938085> 1379640 6 webkit-bug-importer 2017-12-08 10:48:49 -0800 <rdar://problem/35938084> 1380091 7 adam 2017-12-10 16:00:53 -0800 Yes, it does work if we create the audio context when you click a button. https://output.jsbin.com/juzufum Like Andrew said though it would be great if that wasn't the case. 1380673 8 youennf 2017-12-12 09:08:19 -0800 bug 180680 is fixing the case of a page capturing data. Let's keep this bug open for the wider question of allowing web audio analysis without user gesture 1380990 9 adam 2017-12-12 17:57:41 -0800 Fantastic, thanks 1402871 10 daginge 2018-03-01 05:37:57 -0800 We hit this bug as well, where we want to do an automated microphone check for users on their way into a conversation. Our code looks something like this: function handleUserClick() { return mediaDevicesService .getUserMedia(constraints) .then(mediaStream => { this.setState({ mediaStream }); }) .then(() => enumerateDevices()) .then(() => verifyMicrophoneWorks(this.state.mediaStream.stream)) .then(isMicrophoneWorking => this.props.updateMicStatus(isMicrophoneWorking) ) .catch(error => <GetUserMediaErrorFeedback error={error} />); } This all happens in a single promise-chain, but because only the original button click is triggered as a user action, the audio context checking fails, even if all we do is check the getByteFrequencyData, and no audio is actually playing. It would be great if we were allowed to play audio/handle AudioContext once getUserMedia permission is granted for the page for that session. 1402893 11 youennf 2018-03-01 07:41:28 -0800 daginge, I believe that would be addressed in bug 180680. Can you try the latest iOS beta? 1411373 12 daginge 2018-04-03 01:14:36 -0700 Sorry for the late reply youenn. We decided to refactor as more browsers are adopting autoplay restrictions now, and we don't want to take the chance that this causes issues in the future. All of our AudioContext's are triggered by a user action now. 1411415 13 youennf 2018-04-03 08:36:07 -0700 (In reply to daginge from comment #12) > Sorry for the late reply youenn. We decided to refactor as more browsers are > adopting autoplay restrictions now, and we don't want to take the chance > that this causes issues in the future. All of our AudioContext's are > triggered by a user action now. No problem daginge. I believe we could tackle that issue by introducing some WebAudio specific constructs that would allow analyzing audio but not producing any audio. In that case, we could bypass autoplay restrictions. 1411416 14 youennf 2018-04-03 08:45:05 -0700 Maybe OfflineAudioContext is what we want. 1411454 15 youennf 2018-04-03 10:32:02 -0700 (In reply to youenn fablet from comment #14) > Maybe OfflineAudioContext is what we want. Discussed with Jer and this is not designed for this use case. Filed https://github.com/WebAudio/web-audio-api/issues/1551