Bug 218261 – NotSupportedError thrown assigning ConvolverNode buffer

• Status: RESOLVED
• Resolution: INVALID
• Component: Web Audio

[Ashley Gullen]

Demo URL: https://downloads.scirra.com/labs/audio-effects/

Steps to reproduce:
Load the demo, and click 'Add hall convolution'

Observed:
Assigning the 'buffer' property of the convolver node throws 'NotSupportedError: The operation is not supported.'

Expected:
No exception to be thrown. Subsequently playing sound effects should have an audible reverb effect.

It works in both Firefox and Chrome, so it seems only Safari throws on this.

[Chris Dumez]

(In reply to Ashley Gullen from comment #0)
> Demo URL: https://downloads.scirra.com/labs/audio-effects/
> 
> Steps to reproduce:
> Load the demo, and click 'Add hall convolution'
> 
> Observed:
> Assigning the 'buffer' property of the convolver node throws
> 'NotSupportedError: The operation is not supported.'
> 
> Expected:
> No exception to be thrown. Subsequently playing sound effects should have an
> audible reverb effect.
> 
> It works in both Firefox and Chrome, so it seems only Safari throws on this.

Which version of Safari Technology Preview are you testing? I seem unable to reproduce.

[Ashley Gullen]

The latest - TP 115. It also reproduces in the latest production Safari 14.0.

[Chris Dumez]

I have just tried STP 115 (and system Safari 14). I am unable to reproduce the issue with either.

When I click the "Add hall convolution" button, I see:
"Effects added: hallconvolution"

at the top, no error in console. When I click the "Play SFX1" button right after that, I can clearly tell that the sound is different and has an reverb effect.

[Chris Dumez]

(In reply to Chris Dumez from comment #3)
> I have just tried STP 115 (and system Safari 14). I am unable to reproduce
> the issue with either.
> 
> When I click the "Add hall convolution" button, I see:
> "Effects added: hallconvolution"
> 
> at the top, no error in console. When I click the "Play SFX1" button right
> after that, I can clearly tell that the sound is different and has an reverb
> effect.

Looking at our implementation, there are only 3 cases where we may throw a NotSupportedError:
- Sample rate of the buffer does not match sample rate of the audio context
- Buffer does not have 1, 2 or 4 channels
- Buffer has zero size

In WebKit trunk, the exception message would actually give you a proper exception message so you'd know which one is the issue. Sadly, this change is not present in the latest STP yet.

[Ashley Gullen]

I think it's the first reason (different sample rate). In Chrome, Firefox and Safari I get an AudioContext running at 44.1KHz (the default, we don't specify a sample rate). In Chrome and Firefox the audio buffer we assign to the convolver node is also 44.1KHz, presumably because it came from the AudioContext's decodeAudioData which knew to resample it. However our audio files are all WebM Opus - it's nearly supported in all browsers, but Safari is the last browser that does not support this format. So we have to ship a WebAssembly WebM Opus decoder just for Safari. But this decoder outputs audio buffers at a fixed sample rate of 48 KHz, hence the mismatch. (I guess your system's sample rate defaulted to 48 KHz so it matched?)

Does the spec say the sample rate has to match? It doesn't seem to be a problem anywhere else in the Web Audio API, just with this specific case.

While I'm here, please, please add support for WebM Opus. I first asked for this back in 2018 in issue 183852. It would save us so, so much trouble if we didn't have to bundle a decoder just for Safari. And it looks like Safari has been shipping an Opus codec for WebRTC since 2017 anyway. It doesn't seem like such a big jump to hook that up to the Web Audio API.

[Chris Dumez]

(In reply to Ashley Gullen from comment #5)
> I think it's the first reason (different sample rate). In Chrome, Firefox
> and Safari I get an AudioContext running at 44.1KHz (the default, we don't
> specify a sample rate). In Chrome and Firefox the audio buffer we assign to
> the convolver node is also 44.1KHz, presumably because it came from the
> AudioContext's decodeAudioData which knew to resample it. However our audio
> files are all WebM Opus - it's nearly supported in all browsers, but Safari
> is the last browser that does not support this format. So we have to ship a
> WebAssembly WebM Opus decoder just for Safari. But this decoder outputs
> audio buffers at a fixed sample rate of 48 KHz, hence the mismatch. (I guess
> your system's sample rate defaulted to 48 KHz so it matched?)
> 
> Does the spec say the sample rate has to match? It doesn't seem to be a
> problem anywhere else in the Web Audio API, just with this specific case.

Yes, it does:
https://webaudio.github.io/web-audio-api/#dom-convolvernode-buffer

"""
When setting the buffer attribute, execute the following steps synchronously:
If the buffer number of channels is not 1, 2, 4, or if the sample-rate of the buffer is not the same as the sample-rate of its associated BaseAudioContext, a NotSupportedError MUST be thrown.
"""

I guess I am not getting the exception locally because you are not explicitly setting the sample rate of the AudioContext and so I end up with the preferred sample rate of my audio device, which happens to match?


> While I'm here, please, please add support for WebM Opus. I first asked for
> this back in 2018 in issue 183852. It would save us so, so much trouble if
> we didn't have to bundle a decoder just for Safari. And it looks like Safari
> has been shipping an Opus codec for WebRTC since 2017 anyway. It doesn't
> seem like such a big jump to hook that up to the Web Audio API.

Please file a separate bug for this. I am not the right person to look into this particular issue.

[Chris Dumez]

(In reply to Chris Dumez from comment #6)
> (In reply to Ashley Gullen from comment #5)
> > I think it's the first reason (different sample rate). In Chrome, Firefox
> > and Safari I get an AudioContext running at 44.1KHz (the default, we don't
> > specify a sample rate). In Chrome and Firefox the audio buffer we assign to
> > the convolver node is also 44.1KHz, presumably because it came from the
> > AudioContext's decodeAudioData which knew to resample it. However our audio
> > files are all WebM Opus - it's nearly supported in all browsers, but Safari
> > is the last browser that does not support this format. So we have to ship a
> > WebAssembly WebM Opus decoder just for Safari. But this decoder outputs
> > audio buffers at a fixed sample rate of 48 KHz, hence the mismatch. (I guess
> > your system's sample rate defaulted to 48 KHz so it matched?)
> > 
> > Does the spec say the sample rate has to match? It doesn't seem to be a
> > problem anywhere else in the Web Audio API, just with this specific case.
> 
> Yes, it does:
> https://webaudio.github.io/web-audio-api/#dom-convolvernode-buffer
> 
> """
> When setting the buffer attribute, execute the following steps synchronously:
> If the buffer number of channels is not 1, 2, 4, or if the sample-rate of
> the buffer is not the same as the sample-rate of its associated
> BaseAudioContext, a NotSupportedError MUST be thrown.
> """

Interestingly, Chrome has the exact same check as us:
https://github.com/chromium/chromium/blob/a5484e6310a38223fde757b6f094a673ce032cc0/third_party/blink/renderer/modules/webaudio/convolver_node.cc#L117

I am therefore surprised it is throwing in Safari but not Chrome. Maybe the AudioContext ends up with the different sample rate on Safari and Chrome?

[Chris Dumez]

(In reply to Chris Dumez from comment #7)
> (In reply to Chris Dumez from comment #6)
> > (In reply to Ashley Gullen from comment #5)
> > > I think it's the first reason (different sample rate). In Chrome, Firefox
> > > and Safari I get an AudioContext running at 44.1KHz (the default, we don't
> > > specify a sample rate). In Chrome and Firefox the audio buffer we assign to
> > > the convolver node is also 44.1KHz, presumably because it came from the
> > > AudioContext's decodeAudioData which knew to resample it. However our audio
> > > files are all WebM Opus - it's nearly supported in all browsers, but Safari
> > > is the last browser that does not support this format. So we have to ship a
> > > WebAssembly WebM Opus decoder just for Safari. But this decoder outputs
> > > audio buffers at a fixed sample rate of 48 KHz, hence the mismatch. (I guess
> > > your system's sample rate defaulted to 48 KHz so it matched?)
> > > 
> > > Does the spec say the sample rate has to match? It doesn't seem to be a
> > > problem anywhere else in the Web Audio API, just with this specific case.
> > 
> > Yes, it does:
> > https://webaudio.github.io/web-audio-api/#dom-convolvernode-buffer
> > 
> > """
> > When setting the buffer attribute, execute the following steps synchronously:
> > If the buffer number of channels is not 1, 2, 4, or if the sample-rate of
> > the buffer is not the same as the sample-rate of its associated
> > BaseAudioContext, a NotSupportedError MUST be thrown.
> > """
> 
> Interestingly, Chrome has the exact same check as us:
> https://github.com/chromium/chromium/blob/
> a5484e6310a38223fde757b6f094a673ce032cc0/third_party/blink/renderer/modules/
> webaudio/convolver_node.cc#L117
> 
> I am therefore surprised it is throwing in Safari but not Chrome. Maybe the
> AudioContext ends up with the different sample rate on Safari and Chrome?

On my machine:
(new AudioContext).sampleRate returns 48000

on both Chrome and Safari. If your AudioBuffer uses 48000, then that would explain why I did not get an exception on my machine.

[Ashley Gullen]

I get the same audio context sample rate in Safari, Firefox and Chrome. The only difference is the buffer sample rate, for the reasons I explained.

> Please file a separate bug for this. I am not the right person to look into this particular issue.

I did, two years ago, and it appears to have been completely ignored.

It's great that you're doing all this work to make the Web Audio API more interoperable, but from my point of view, this all has less effect than supporting the same codecs as all other browsers do. The fact Safari can't play the same formats as all other browsers is a huge interoperability barrier and causes a catalog of problems, this being just the latest.

[Chris Dumez]

(In reply to Chris Dumez from comment #7)
> (In reply to Chris Dumez from comment #6)
> > (In reply to Ashley Gullen from comment #5)
> > > I think it's the first reason (different sample rate). In Chrome, Firefox
> > > and Safari I get an AudioContext running at 44.1KHz (the default, we don't
> > > specify a sample rate). In Chrome and Firefox the audio buffer we assign to
> > > the convolver node is also 44.1KHz, presumably because it came from the
> > > AudioContext's decodeAudioData which knew to resample it. However our audio
> > > files are all WebM Opus - it's nearly supported in all browsers, but Safari
> > > is the last browser that does not support this format. So we have to ship a
> > > WebAssembly WebM Opus decoder just for Safari. But this decoder outputs
> > > audio buffers at a fixed sample rate of 48 KHz, hence the mismatch. (I guess
> > > your system's sample rate defaulted to 48 KHz so it matched?)
> > > 
> > > Does the spec say the sample rate has to match? It doesn't seem to be a
> > > problem anywhere else in the Web Audio API, just with this specific case.
> > 
> > Yes, it does:
> > https://webaudio.github.io/web-audio-api/#dom-convolvernode-buffer
> > 
> > """
> > When setting the buffer attribute, execute the following steps synchronously:
> > If the buffer number of channels is not 1, 2, 4, or if the sample-rate of
> > the buffer is not the same as the sample-rate of its associated
> > BaseAudioContext, a NotSupportedError MUST be thrown.
> > """
> 
> Interestingly, Chrome has the exact same check as us:
> https://github.com/chromium/chromium/blob/
> a5484e6310a38223fde757b6f094a673ce032cc0/third_party/blink/renderer/modules/
> webaudio/convolver_node.cc#L117
> 
> I am therefore surprised it is throwing in Safari but not Chrome. Maybe the
> AudioContext ends up with the different sample rate on Safari and Chrome?

I have also verified by running Chrome that they have the same check:

ac = new AudioContext({ sampleRate: 48000 });
> AudioContext {baseLatency: 0.005333333333333333, destination: AudioDestinationNode, currentTime: 0, sampleRate: 48000, listener: AudioListener, …}

buffer = new AudioBuffer({ sampleRate: 44000, length: 100 });
> AudioBuffer {length: 100, duration: 0.0022727272727272726, sampleRate: 44000, numberOfChannels: 1}
conv = ac.createConvolver()
> ConvolverNode {buffer: null, normalize: true, context: AudioContext, numberOfInputs: 1, numberOfOutputs: 1, …}
conv.buffer = buffer
VM902:1 Uncaught DOMException: Failed to set the 'buffer' property on 'ConvolverNode': The buffer sample rate of 44000 does not match the context rate of 48000 Hz.
    at <anonymous>:1:13
(anonymous) @ VM902:1

[Chris Dumez]

(In reply to Ashley Gullen from comment #9)
> I get the same audio context sample rate in Safari, Firefox and Chrome. The
> only difference is the buffer sample rate, for the reasons I explained.
> 
> > Please file a separate bug for this. I am not the right person to look into this particular issue.
> 
> I did, two years ago, and it appears to have been completely ignored.
> 
> It's great that you're doing all this work to make the Web Audio API more
> interoperable, but from my point of view, this all has less effect than
> supporting the same codecs as all other browsers do. The fact Safari can't
> play the same formats as all other browsers is a huge interoperability
> barrier and causes a catalog of problems, this being just the latest.

I feel your pain when it comes to codecs. I cc'd Youenn, Jer & Eric who may be familiar with this issue as this is really not my area.

[Chris Dumez]

On this particular bug though, it seems our behavior matches the specification and Firefox. Can I close this bug or do you still feel there is a WebKit bug here?

Note that I appreciate you testing our new WebAudio support in STP and reporting issues :)

[Ashley Gullen]

Thanks for your help figuring out what was going on here, I think it can be closed as it's a sample rate mismatch as per the spec.

The root cause is it's difficult to decode audio with interoperable sample rates, because Safari doesn't support the same audio formats that all other browsers do, so we can't use decodeAudioData. I filed issue 183852 over two years ago about that, specifically in the hopes we'd avoid these kinds of interoperability headaches, but there has been no response at all on that. If anyone at Apple is reading this: please support WebM Opus - it would make a bigger improvement to interoperability than all this work on modernising the Web Audio API. (That's still valuable work I'd add, it's just the codec support is a bigger hurdle than some API differences.)

[Chris Dumez]

Thanks. I cc'd appropriate people on Bug 183852.

[Chris Dumez]

(In reply to Ashley Gullen from comment #5)
> I think it's the first reason (different sample rate). In Chrome, Firefox
> and Safari I get an AudioContext running at 44.1KHz (the default, we don't
> specify a sample rate). In Chrome and Firefox the audio buffer we assign to
> the convolver node is also 44.1KHz, presumably because it came from the
> AudioContext's decodeAudioData which knew to resample it. However our audio
> files are all WebM Opus - it's nearly supported in all browsers, but Safari
> is the last browser that does not support this format. So we have to ship a
> WebAssembly WebM Opus decoder just for Safari. But this decoder outputs
> audio buffers at a fixed sample rate of 48 KHz, hence the mismatch. (I guess
> your system's sample rate defaulted to 48 KHz so it matched?)
> 
> Does the spec say the sample rate has to match? It doesn't seem to be a
> problem anywhere else in the Web Audio API, just with this specific case.
> 
> While I'm here, please, please add support for WebM Opus. I first asked for
> this back in 2018 in issue 183852. It would save us so, so much trouble if
> we didn't have to bundle a decoder just for Safari. And it looks like Safari
> has been shipping an Opus codec for WebRTC since 2017 anyway. It doesn't
> seem like such a big jump to hook that up to the Web Audio API.

Bug 221745 may be of interest to you.