Bug 83558 - Add Doppler effect support for MediaElementAudioSourceNode
Summary: Add Doppler effect support for MediaElementAudioSourceNode
Status: UNCONFIRMED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-04-10 02:24 PDT by Xingnan Wang
Modified: 2012-04-19 00:21 PDT (History)
4 users (show)

See Also:


Attachments
Patch (6.67 KB, patch)
2012-04-10 02:39 PDT, Xingnan Wang
crogers: review-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Xingnan Wang 2012-04-10 02:24:27 PDT
AudioBufferSourceNode will incorporate the doppler shift to pitch rate, MediaElementAudioSourceNode should also support doppler effect on the audio source.
Comment 1 Xingnan Wang 2012-04-10 02:39:11 PDT
Created attachment 136415 [details]
Patch
Comment 2 Eric Carlson 2012-04-10 09:20:26 PDT
Is this requirement in the spec?
Comment 3 Chris Rogers 2012-04-10 10:38:05 PDT
(In reply to comment #2)
> Is this requirement in the spec?

It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.
Comment 4 Xingnan Wang 2012-04-18 00:58:28 PDT
(In reply to comment #3)
> (In reply to comment #2)
> > Is this requirement in the spec?
> 
> It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.

Hi Chris,

I`m a little confused that AudioBufferSourceNode supports the doppler effect but HTMLMediaElement not.
They are both audio sources, why different?
Does that mean HTMLMediaElementNode + Panner(with doppler) is rare case and it`s meaningless to support it?
Also it seems that they are not specified in spec. 
And how about new-added Oscillator? It should be another kind of source node, right?
Comment 5 Chris Rogers 2012-04-18 10:35:18 PDT
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Is this requirement in the spec?
> > 
> > It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.
> 
> Hi Chris,
> 
> I`m a little confused that AudioBufferSourceNode supports the doppler effect but HTMLMediaElement not.
> They are both audio sources, why different?

Yes, this is a good point.  But it is difficult to implement the pitch shift using this technique as in this patch.


> Does that mean HTMLMediaElementNode + Panner(with doppler) is rare case and it`s meaningless to support it?

I think it will be a much rarer case than AudioBufferSourceNode.  I wouldn't say it's meaningless in an ideal world, but it is just extremely fragile to try to pitch shift with resampler changes in this way because of the way the HTMLMediaElements are implemented in browsers.

> Also it seems that they are not specified in spec. 

True, I should be clear about it.

> And how about new-added Oscillator? It should be another kind of source node, right?

I think the Oscillator can support the pitch shift very similar to AudioBufferSourceNode.  I'll try to be clear in the spec...
Comment 6 Xingnan Wang 2012-04-19 00:21:00 PDT
(In reply to comment #5)
> (In reply to comment #4)
> > (In reply to comment #3)
> > > (In reply to comment #2)
> > > > Is this requirement in the spec?
> > > 
> > > It isn't in the spec, and I wasn't anticipating this as even a possibility.  I would suggest we avoid this because it could significantly complicate integration with HTMLMediaElement.  I also don't think it's likely to be needed very often.
> > 
> > Hi Chris,
> > 
> > I`m a little confused that AudioBufferSourceNode supports the doppler effect but HTMLMediaElement not.
> > They are both audio sources, why different?
> 
> Yes, this is a good point.  But it is difficult to implement the pitch shift using this technique as in this patch.
> 
> 
> > Does that mean HTMLMediaElementNode + Panner(with doppler) is rare case and it`s meaningless to support it?
> 
> I think it will be a much rarer case than AudioBufferSourceNode.  I wouldn't say it's meaningless in an ideal world, but it is just extremely fragile to try to pitch shift with resampler changes in this way because of the way the HTMLMediaElements are implemented in browsers.

Do you mean the real limitation is we use resampler to change the pitch? 
It will be much better to use other pitch-shift methods without time change, right?

> 
> > Also it seems that they are not specified in spec. 
> 
> True, I should be clear about it.
> 
> > And how about new-added Oscillator? It should be another kind of source node, right?
> 
> I think the Oscillator can support the pitch shift very similar to AudioBufferSourceNode.  I'll try to be clear in the spec...

I also have another question. We implement resampling internally in AudioBufferSourceNode, and do doppler pitch shifting together now. But why not to implement doppler pitch shift separately in pannerNode? 
One possible reason I guess is it may reduce once resamping, but if so how about the case that we have many AudioBufferSourceNode?