Bug 183230

Summary: Web Automation: script evaluations via WebDriver should have a user gesture indicator
Product: WebKit Reporter: BJ Burg <bburg>
Component: WebDriverAssignee: BJ Burg <bburg>
Status: RESOLVED FIXED    
Severity: Normal CC: aestes, bburg, bfulgham, cgarcia, darin, joepeck, thorton, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Proposed Fix aestes: review+

Description BJ Burg 2018-02-28 15:57:15 PST 
Some JS/DOM APIs only work require a user gesture indicator. Since there is no real user input when running via WebDriver, these APIs can only be tested if WebDriver code has a synthetic user gesture indicator, or explicit checks for a user gesture indicator should also allow usage iff page->isControlledByAutomation().

It would be nice to do the first approach since only one source location is needed. However, it's possible via WebDriver to evaluate "async" JavaScript which calls a completion handler some arbitrary amount of time later. The naive approach to set a user gesture indicator would only allow gesture-requiring API calls that happen synchronously.

So, we need to either set a user gesture indicator using non-RAII code, or audit every single check and add || page->isControlledByAutomation() to the condition.
Comment 1 BJ Burg 2018-02-28 15:58:58 PST 
<rdar://problem/37959739>
Comment 2 BJ Burg 2018-02-28 16:06:52 PST 
(In reply to Brian Burg from comment #0)
> Some JS/DOM APIs only work require a user gesture indicator. Since there is
> no real user input when running via WebDriver, these APIs can only be tested
> if WebDriver code has a synthetic user gesture indicator, or explicit checks
> for a user gesture indicator should also allow usage iff
> page->isControlledByAutomation().
> 
> It would be nice to do the first approach since only one source location is
> needed. However, it's possible via WebDriver to evaluate "async" JavaScript
> which calls a completion handler some arbitrary amount of time later. The
> naive approach to set a user gesture indicator would only allow
> gesture-requiring API calls that happen synchronously.
> 
> So, we need to either set a user gesture indicator using non-RAII code, or
> audit every single check and add || page->isControlledByAutomation() to the
> condition.

Actually, it seems like DOMTimer and some other async code will forward the user gesture, so maybe using the UserGestureIndicator when evaluating the JS snippet would be enough.
Comment 3 BJ Burg 2018-03-02 20:39:28 PST 
Created attachment 334948 [details]
Proposed Fix
Comment 4 BJ Burg 2018-03-04 14:59:24 PST 
Committed r229206: <https://trac.webkit.org/changeset/229206>