Bug 136294 - [META] Web Replay: capture and replay user actions from WebPage instead of UserInputBridge
Summary: [META] Web Replay: capture and replay user actions from WebPage instead of Us...
Status: RESOLVED LATER
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit2 (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on: 136312 140445 140446 140447 140448 140451 140452 140453 140454
Blocks: WebReplay 129694
  Show dependency treegraph
 
Reported: 2014-08-27 10:23 PDT by Brian Burg
Modified: 2017-07-10 13:59 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Burg 2014-08-27 10:23:17 PDT
There are some nondeterministic things that can only be captured and replayed correctly from WebKit2, such as a page's BackForwardList (https://webkit.org/b/131043), ViewState (https://webkit.org/b/129694), initial frame size (https://webkit.org/b/131337), initial focus, and probably other things.

To move towards this, we can move support of some existing nondeterministic things from WebCore to WebKit2. Navigation events (reload, stop, load; https://webkit.org/b/129447) and user events (mouse, keyboard, wheel, touch, etc) currently go through UserInputBridge before hitting FocusController/EventHandler/Page in WebCore. I would like to remove this layer of indirection by moving the capture/replay code paths in UserInputBridge to WebPage. I have this approach working in my local repository and will clean it up for review in the next weeks.

There is some additional work to make it possible to create/dispatch replayable inputs from WebKit2. I'll attach separate bugs for those items.
Comment 1 Brian Burg 2015-01-14 10:01:27 PST
This is a tracking bug.

Currently, web replay captures and re-dispatches user actions such as mouse, keyboard, navigations, viewState changes in WebCore. However, in many cases it requires fewer and simpler hooks to do this work in WebPage. For example, we can capture one type of mouse action (MouseEvent) rather than 3+ (Press, Release, Move, FakeMove).

There are other web features that need WebKit2 support to be replayed correctly:
 * capturing and restoring script-accessible history [https://bugs.webkit.org/show_bug.cgi?id=131043]
 * capturing and replaying history navigations [https://bugs.webkit.org/show_bug.cgi?id=131084]
 * capturing and replaying view state changes that script can see [https://bugs.webkit.org/show_bug.cgi?id=129694]
 * persistent state that's not stored/managed in WebCore

This requires some changes to the replay code generator so some inputs can dispatch from a WebPage& instead of a Page&. There is also a small change to control flow for user inputs, to essentially inline the functionality of UserInputBridge methods into its callsites.

Current control flow:

[UIProcess] WebPageProxy::handleMouseEvent
 -> [WebProcess] WebPage::mouseEvent
 -> [WebCore] UserInputBridge::handleMouseEvent // save action data if capturing, or simulate action by calling during playback
 -> [WebCore] EventHandler::handleMouseEvent

Proposed control flow:

[UIProcess] WebPageProxy::handleMouseEvent
 -> [WebProcess] WebPage::mouseEvent // forward to handleMouseEvent, then send IPC reply
 -> [WebProcess] WebPage::handleMouseEvent // save action data if capturing, or simulate action by calling during playback
 -> [WebCore] EventHandler::handleMouseEvent
Comment 2 Brian Burg 2017-07-10 13:59:29 PDT
Closing web replay-related bugs until we resume working on the feature again.