Bug 83558 – Add Doppler effect support for MediaElementAudioSourceNode

• Component: Web Audio
• Status: UNCONFIRMED

[Xingnan Wang]

AudioBufferSourceNode will incorporate the doppler shift to pitch rate, MediaElementAudioSourceNode should also support doppler effect on the audio source.

[Xingnan Wang]

Created attachment 136415 [details]
Patch

[Eric Carlson]

Is this requirement in the spec?

[Chris Rogers]

(In reply to comment #2)
> Is this requirement in the spec?

It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.

[Xingnan Wang]

(In reply to comment #3)
> (In reply to comment #2)
> > Is this requirement in the spec?
> 
> It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.

Hi Chris,

I`m a little confused that AudioBufferSourceNode supports the doppler effect but HTMLMediaElement not.
They are both audio sources, why different?
Does that mean HTMLMediaElementNode + Panner(with doppler) is rare case and it`s meaningless to support it?
Also it seems that they are not specified in spec. 
And how about new-added Oscillator? It should be another kind of source node, right?

[Chris Rogers]

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Is this requirement in the spec?
> > 
> > It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.
> 
> Hi Chris,
> 
> I`m a little confused that AudioBufferSourceNode supports the doppler effect but HTMLMediaElement not.
> They are both audio sources, why different?

Yes, this is a good point.  But it is difficult to implement the pitch shift using this technique as in this patch.


> Does that mean HTMLMediaElementNode + Panner(with doppler) is rare case and it`s meaningless to support it?

I think it will be a much rarer case than AudioBufferSourceNode.  I wouldn't say it's meaningless in an ideal world, but it is just extremely fragile to try to pitch shift with resampler changes in this way because of the way the HTMLMediaElements are implemented in browsers.

> Also it seems that they are not specified in spec. 

True, I should be clear about it.

> And how about new-added Oscillator? It should be another kind of source node, right?

I think the Oscillator can support the pitch shift very similar to AudioBufferSourceNode.  I'll try to be clear in the spec...

[Xingnan Wang]

(In reply to comment #5)
> (In reply to comment #4)
> > (In reply to comment #3)
> > > (In reply to comment #2)
> > > > Is this requirement in the spec?
> > > 
> > > It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.
> > 
> > Hi Chris,
> > 
> > I`m a little confused that AudioBufferSourceNode supports the doppler effect but HTMLMediaElement not.
> > They are both audio sources, why different?
> 
> Yes, this is a good point.  But it is difficult to implement the pitch shift using this technique as in this patch.
> 
> 
> > Does that mean HTMLMediaElementNode + Panner(with doppler) is rare case and it`s meaningless to support it?
> 
> I think it will be a much rarer case than AudioBufferSourceNode.  I wouldn't say it's meaningless in an ideal world, but it is just extremely fragile to try to pitch shift with resampler changes in this way because of the way the HTMLMediaElements are implemented in browsers.

Do you mean the real limitation is we use resampler to change the pitch? 
It will be much better to use other pitch-shift methods without time change, right?

> 
> > Also it seems that they are not specified in spec. 
> 
> True, I should be clear about it.
> 
> > And how about new-added Oscillator? It should be another kind of source node, right?
> 
> I think the Oscillator can support the pitch shift very similar to AudioBufferSourceNode.  I'll try to be clear in the spec...

I also have another question. We implement resampling internally in AudioBufferSourceNode, and do doppler pitch shifting together now. But why not to implement doppler pitch shift separately in pannerNode? 
One possible reason I guess is it may reduce once resamping, but if so how about the case that we have many AudioBufferSourceNode?