Bug 174078 – Start audio inside non-usergesture method

• Component: Web Audio
• Status: NEW

[Andrei Yangabishev]

Created attachment 314417 [details]
htm/js/jpg/mp3/diff.path with additional logging entries

I am trying to use ogv.js in my project and found case when I cannot run ogg video after downloading a list of files. I download list of files, select less expensive with lowest resolution and try to play him but ogv.js could not play him because WebAudio doesn't read buffer. I downloaded WebKit master branch and found than AudioContext::userGestureRequiredForAudioStart prevents to play audio because I try to play audio inside XMLHttpRequest::onLoad method (it is not user gesture method).

Really I had two methods. I invoke AudioContext::createBufferSource in onclick method but start audio in XMLHttpRequest::onLoad and it works. The second case is when I createBufferSource is onLoad and it doesn't work, no audio.

I have created test case when I createBufferSource and invoke start method inside onLoad. I have not found any info what rules are used to run audio from JS, not using by <video/audio> html elements. Am I right that if I started audio inside user gesture method oneday then it enables to run audio from all non user gesture methods like timers and onLoad?

I used b16afe0e39a557007693a5e5dcebdc1fe51141ff commit as start point for investigation.

////////////////

var audioCtx = new (window.AudioContext || window.webkitAudioContext);

function playMusic(file) {
    var request  = new XMLHttpRequest();
    request.open('GET', file);
    request.responseType = 'arraybuffer';
    request.onload = function() {
        var audioData = request.response;
        audioCtx.decodeAudioData(audioData, function(buffer) {
            console.log('decode');
            var source = audioCtx.createBufferSource(); // !!! Am I right I should to move it inside playMusic(onClick) method?
            source.buffer = buffer;
            source.connect(audioCtx.destination);
            source.start(0);
        }, function(e){console.log("Error with decoding audio data" + e.err)});
    };
    request.send();
}

// here I close second banner and run audio since 5 seconds from timer and(or) XMLHttpRequest::onLoad
document.getElementById('b').addEventListener('click',function(){
    document.getElementById('b').style.display = 'none';
    var source = audioCtx.createBufferSource();
    source.start(0);
    setTimeout(playMusic.bind(null,'b.mp3'),5000);
});

////////////////

LOG

AudioContext::constructCommon (line 2)
AudioContext::constructCommon::add RequireUserGestureForAudioStartRestriction (line 2)
AudioContext::AudioContext::create default audio destination node (line 2)
AudioContext::createBufferSource::create audio buffer source node (line 23)
AudioContext::Initialize (line 23)
AudioContext::Initialize::initialize destination node (line 23)
AudioContext::Initialize::invoke start rendering (line 23)
AudioContext::startRendering (line 23)
AudioContext::willBeginPlayback (line 23)
AudioContext::willBeginPlayback::userGestureRequiredForAudioStart = true (line 23)
Document::processingUserGestureForMedia::ScriptController::processingUserGestureForMedia=true -> return true (line 23)
AudioContext::processingUserGestureForMedia::has document and document::processingUserGestureForMedia has returned true (line 23)
AudioContext::willBeginPlayback::remove RequireUserGestureForAudioStartRestriction (line 23)
AudioContext::willBeginPlayback::return (line 23)
AudioContext::startRendering::willBeginPlayback has returned true -> destination::startRendering (line 23)
AudioBufferSourceNode::start (line 24)
AudioBufferSourceNode::startPlaying (line 24)
AudioContext::nodeWillBeginPlayback (line 24)
AudioContext::nodeWillBeginPlayback::skip context::startRendering because userGestureRequiredForAudioStart=false (line 24)
AudioContext::decodeAudioData (line 10)
decode (line 11)
AudioContext::createBufferSource::create audio buffer source node (line 12)
AudioBufferSourceNode::start (line 15)
AudioBufferSourceNode::startPlaying (line 15)
AudioContext::nodeWillBeginPlayback (line 15)
AudioContext::nodeWillBeginPlayback::skip context::startRendering because userGestureRequiredForAudioStart=false (line 15)

[Andrei Yangabishev]

tested on Safari 10.0 (12602.1.50.0.10) and iOS simulator iPhone 5 9.3

[Radar WebKit Bug Importer]

<rdar://problem/33102855>

[Jer Noble]

Andrei, you would have to start the AudioContext (with resume()) on a user gesture.  That method is called automatically when the first audio node is connected to a destination.  But you can also just call it on it's own during a gesture.