Bug 322023 – [Web Automation] Element coordinates in cross-origin iframes under Site Isolation omit the page scale and layout viewport offset

• Status: NEW
• Component: WebDriver

[Razvan Caliman]

Follow-up to bug 321593, which finished the LayoutViewport coordinate conversion for
out-of-process iframes in the UI process. That conversion is incomplete.

When a frame's local root is the main frame, WebAutomationSessionProxy::computeElementLayout()
produces LayoutViewport coordinates by applying LocalFrameView::rootViewToContents() followed by
FrameView::absoluteToLayoutViewportRect(). Together those apply four terms beyond the frame
offset:

- division by Frame::frameScaleFactor()               (FrameView.cpp, absoluteToLayoutViewport*)
- subtraction of FrameView::layoutViewportRect().location()
- subtraction of ScrollView::headerHeight()           (ScrollView.cpp, rootViewToContents)
- subtraction of ScrollView::insetForLeftScrollbarSpace()

Under Site Isolation the frame's web process stops at local root contents coordinates, and
WebAutomationSession walks them up to the main frame with convertRectToMainFrameCoordinates() /
convertPointToMainFrameCoordinates() and then subtracts the obscured content insets. All four
terms above are dropped.

They cancel out when the frame scale is 1, the layout viewport origin equals the scroll position,
and there is no header banner or left-hand scrollbar. That covers every case the WebDriver test
suite currently exercises, which is why the tests pass. Outside those conditions the reported
coordinates are wrong:

- page scale != 1 (pinch zoom, unless the page delegates scaling): off by roughly the page
  scale factor. Element Click synthesizes the pointer outside the element and the click is lost.
- layout viewport origin != scroll position: residual visual-viewport offset.
- header banner installed: off by the banner height.
- RTL page with the scrollbar on the left: off by the scrollbar width.

Neither side of the existing conversion can supply the missing terms. The main frame's
pageScaleFactor() and layoutViewportRect() live in the main frame's web process, and an
out-of-process subframe cannot see them. Frame::frameScaleFactor() returns the frame's CSS zoom
rather than the page scale for a subframe (Frame.cpp). Replicating the arithmetic in the UI
process would duplicate documentScrollPositionRelativeToViewOrigin() and the layout viewport
logic outside WebCore.

Proposed fix: add a WebPage message that asks the main frame's process to apply
absoluteToLayoutViewportRect()/Point() to the converted coordinates, so the site-isolated path
computes the same expression as the non-isolated one by construction rather than by matching term
lists. That handles all four terms at once and lets obscuredContentInsetOffset() in
WebAutomationSession.cpp be deleted.

Both ComputeElementLayout consumers need it: WebAutomationSession::computeElementLayout() and
WebAutomationSession::viewportInViewCenterPointOfElement(), the latter being the one that
positions clicks.

The assumption is documented at the subtraction site in WebAutomationSession.cpp; this bug tracks
removing it.

[Radar WebKit Bug Importer]

<rdar://problem/185214089>