Bug 128150 - Web Replay: route through UserInputBridge when delivering user inputs to WebCore
Summary: Web Replay: route through UserInputBridge when delivering user inputs to WebCore
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit2 (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: BJ Burg
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-02-03 18:28 PST by BJ Burg
Modified: 2014-02-26 13:35 PST (History)
16 users (show)

See Also:


Attachments
patch (27.53 KB, patch)
2014-02-03 18:57 PST, BJ Burg
no flags Details | Formatted Diff | Diff
patch2 (31.67 KB, patch)
2014-02-04 10:30 PST, BJ Burg
no flags Details | Formatted Diff | Diff
rebased patch (31.00 KB, patch)
2014-02-10 10:01 PST, BJ Burg
no flags Details | Formatted Diff | Diff
rebased, renamed to UserInputBridge (33.19 KB, patch)
2014-02-22 15:41 PST, BJ Burg
no flags Details | Formatted Diff | Diff
try EWS again (33.89 KB, patch)
2014-02-24 11:08 PST, BJ Burg
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description BJ Burg 2014-02-03 18:28:04 PST
This is one of the few parts of web replay that cannot be easily put behind a guard. For now, the methods inside InputProxy don't have any replay code paths, as that requires machinery that hasn't landed yet.
Comment 1 BJ Burg 2014-02-03 18:57:48 PST
Created attachment 223059 [details]
patch
Comment 2 BJ Burg 2014-02-03 19:00:33 PST
RFC: should this patch forward user inputs for WK2 and PLATFORM(MAC) only, or should I go dig for all uses of the user input entry points? I could see the use of replaying Mac WK1, but for other ports it seems like a waste of time unless web replay is intended to support those platforms (or unless InputProxy becomes useful to non-replay code for refactoring reasons).
Comment 3 BJ Burg 2014-02-04 10:30:15 PST
Created attachment 223137 [details]
patch2
Comment 4 BJ Burg 2014-02-10 10:01:53 PST
Created attachment 223721 [details]
rebased patch
Comment 5 Andreas Kling 2014-02-10 10:08:22 PST
I'm very excited about this feature, but we need to talk about the abstraction layer added here (CC'ing people.)
Comment 6 BJ Burg 2014-02-10 10:12:16 PST
(In reply to comment #5)
> I'm very excited about this feature, but we need to talk about the abstraction layer added here (CC'ing people.)

Sure thing. (For reference, an older version of InputProxy.cpp with the replay parts added is viewable here: https://github.com/burg/timelapse/blob/timelapse/Source/WebCore/replay/ReplayProxy.cpp)
Comment 7 BJ Burg 2014-02-19 18:05:06 PST
Update:

After much bikeshedding over the last few weeks, I think the name UserInputBridge or UserInputSwitch best communicate what this abstraction does: It lets inputs selectively flow from WK/WK2 to WebCore. 

When capturing, the inputs are tee'd into the replay log and EventHandler/FocusController/wherever they used to go. When replaying, the 'bridge' is up (or, circuit broken) and user inputs don't come from WK/WK2, only from the replay log and internally.

These names avoid the following overloaded/vague terms: Shim, Proxy, Mediator, Connection, Channel. I think some others have suggested Switchboard and InputTap, but those are less evocative IMO.


As for the necessity of adding yet another abstraction, the alternatives are less appealing because they distribute 'bridge' logic across many callsites in WebKit2 and/or many callees in WebCore.

Alternative 1: inside every relevant input-receiving callee in WebCore, add:

 * an optional argument to distinguish the source of the input (WC or WK2)
 * check whether we are capturing or replaying.
   - If capturing, save the user input if it's external (from WK2).
   - If replaying, ignore the user input if it's external (from WK2), and handle normally otherwise (from recording or WebCore).

Alternative 2: at every relevant input-delivering callsite in WebProcess, implement the above, but only for the external case.

Either alternative requires duplicating the delivery code (e.g., corePage()->focusController()->focusedOrMainFrame()->eventHandler()->handleMouseMove(...)) between the WK2 callsite and the replay input's dispatch() method, which could lead to subtle bitrot.
Comment 8 Timothy Hatcher 2014-02-19 18:17:27 PST
I like UserInputBridge.
Comment 9 BJ Burg 2014-02-22 15:41:28 PST
Created attachment 224984 [details]
rebased, renamed to UserInputBridge
Comment 10 Timothy Hatcher 2014-02-24 09:53:18 PST
Comment on attachment 224984 [details]
rebased, renamed to UserInputBridge

Looks good to me. Please fix the other platform builds before landing though.
Comment 11 BJ Burg 2014-02-24 10:03:11 PST
(In reply to comment #10)
> (From update of attachment 224984 [details])
> Looks good to me. Please fix the other platform builds before landing though.

OK. I haven't been able to get a useful GTK or EFL build attempt in a few weeks though.
Comment 12 BJ Burg 2014-02-24 11:08:08 PST
Created attachment 225075 [details]
try EWS again
Comment 13 Joseph Pecoraro 2014-02-24 11:39:57 PST
Comment on attachment 225075 [details]
try EWS again

View in context: https://bugs.webkit.org/attachment.cgi?id=225075&action=review

Also looks good to me.

> Source/WebCore/replay/UserInputBridge.h:54
> +    bool handleContextMenuEvent(const PlatformMouseEvent&, const Frame*, bool fromReplay = false);
> +    bool handleMousePressEvent(const PlatformMouseEvent&, bool fromReplay = false);
> +    bool handleMouseReleaseEvent(const PlatformMouseEvent&, bool fromReplay = false);
> +    bool handleMouseMoveEvent(const PlatformMouseEvent&, bool fromReplay = false);
> +    bool handleMouseMoveOnScrollbarEvent(const PlatformMouseEvent&, bool fromReplay = false);

When the fromReplay parameter is actually used, it would read better if it was an enum instead of a bool. WebKit style is typically to use an enum instead of booleans where possible to avoid confusing parameter values at call sites. For example: foo(true, false, true);
Comment 14 BJ Burg 2014-02-26 13:35:58 PST
Committed r164745: <http://trac.webkit.org/changeset/164745>