Bug 149367 - REGRESSION: WebAudio user-gesture restriction is no longer lifted by touchstart event
Summary: REGRESSION: WebAudio user-gesture restriction is no longer lifted by touchsta...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: Safari 9
Hardware: iPhone / iPad iOS 9.0
: P2 Normal
Assignee: Jer Noble
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2015-09-18 20:41 PDT by Jay Oster
Modified: 2015-11-09 06:27 PST (History)
14 users (show)

See Also:


Attachments
Patch (1.98 KB, patch)
2015-09-29 11:30 PDT, Jer Noble
no flags Details | Formatted Diff | Diff
Patch (4.99 KB, patch)
2015-09-29 13:10 PDT, Jer Noble
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jay Oster 2015-09-18 20:41:10 PDT
Safari 9 lost the ability to change the AudioContext status to "running" from a touchstart event.

The unit test uses a mousedown event: http://trac.webkit.org/browser/trunk/LayoutTests/webaudio/audiocontext-restriction.html?rev=183424#L45 runWithKeyDown() is called, and: http://trac.webkit.org/browser/trunk/LayoutTests/webaudio/resources/audio-testing.js?rev=183424#L198 mousedown event is triggered.

mousedown and touchend events can be used to lift the restriction. touchstart no longer works.

Possible regression from #144211 ?
Comment 1 Eric Carlson 2015-09-19 13:33:04 PDT
I suspect this was caused by the changes for r186148, which changed the events that cause ScriptController::processingUserGesture to return true.
Comment 2 Ashley Gullen 2015-09-22 03:44:45 PDT
This affects a huge amount of existing web content. All Construct 2 content, and as far as I can tell all Phaser, soundjs and howler.js content is affected, because all these frameworks rely on touchstart to unblock the Web Audio API.

I have made two test cases:
https://www.scirra.com/labs/bugs/iosaudio/touchstart.html - unblocks in touchstart event, only plays audio in iOS 8 and older

https://www.scirra.com/labs/bugs/iosaudio/touchend.html - unblocks in touchend event, works in iOS 9 as well

More discussion here: http://www.html5gamedevs.com/topic/17326-ios-9-webaudio-issues/

I'd estimate most web content in the wild using the Web Audio API now remains silent on iOS 9, basically making it useless. This is a severe regression from iOS 8; us framework developers can distribute workarounds, but we cannot update all the existing broken content out there, Safari needs to be fixed as well.
Comment 3 Ashley Gullen 2015-09-22 04:06:38 PDT
This bug is serious enough that we are now making public announcements to our userbase about its existence and status: https://www.scirra.com/forum/notice-ios-9-breaks-all-audio-playback_t161387
Comment 4 Ashley Gullen 2015-09-22 04:49:00 PDT
This also prevents playback of <audio> and <video> in a touchstart event. It only works in touchend.
Comment 5 Chinmay Pendharkar 2015-09-22 18:37:58 PDT
This is indeed quite an annoying bug. It broke a bunch of libraries which relied on the first user interaction "a touchstart event" to 'unmute' the WebAudio context. This is not only critical for HTML5 game engines but also a ton of WebAudio based websites. Please consider if this is regression. 

If it's an update to the policy on when websites are allowed to create sound, it would be really great if this is updated and documented somewhere.
Comment 6 Eric Carlson 2015-09-23 06:47:02 PDT
<rdar://problem/22695878>
Comment 7 Jer Noble 2015-09-29 11:29:39 PDT
Honestly, "touchend" is the correct event to use for removing the gesture restriction. "touchstart" could be the beginning of a scroll gesture, and playing audio during this gesture could be completely undesirable from a user's perspective.  And in iOS 9, the frameworks underneath WebKit did indeed re-classify "touchstart" as an event which did not indicate an explicit user gesture.

So I would encourage the library authors CCd here to not remove their touchend workarounds. We'll fix this for backwards compatibility, but there may be a future date where audio-on-"touchend" will be a requirement.

(Also, keep in mind that the first explicit user gesture will lift the audio restriction for an entire AudioSession. So, after the first user gesture, audio initiated by "touchstart" events will be audible.)
Comment 8 Jer Noble 2015-09-29 11:30:31 PDT
Created attachment 262083 [details]
Patch
Comment 9 Jer Noble 2015-09-29 13:10:56 PDT
Created attachment 262096 [details]
Patch
Comment 10 WebKit Commit Bot 2015-09-29 14:04:19 PDT
Comment on attachment 262096 [details]
Patch

Clearing flags on attachment: 262096

Committed r190327: <http://trac.webkit.org/changeset/190327>
Comment 11 WebKit Commit Bot 2015-09-29 14:04:24 PDT
All reviewed patches have been landed.  Closing bug.
Comment 12 Jimm 2015-10-09 05:33:57 PDT
I realize I'm a bit late to the party, but have these patches been upstreamed to the iOS 9.1 beta releases yet?  If so, then it appears the bug behavior has not changed - at least on my iPad Air (Model MD786KN/B) with iOS 9.1 beta 3 (13B5130b)
Comment 13 Geoffrey Garen 2015-10-09 09:13:22 PDT
I agree with Jer's earlier comment, and I think we might want to revert here.

Allowing media to play on touchstart means that ads videos and audio will autoplay when you scroll. That's a bad user experience -- and we made an intentional change to stop it in the case of opening windows and apps and other things.