AX: fix coordinate mapping for iOS accessibility

• Component: Accessibility
• Status: RESOLVED FIXED

[chris fleizach]

The conversion routines that accessibility is using to convert from rootToScreen and back are not "scene reference" aware.

That means those methods convert to the screen space, but VoiceOver and others are expecting it in scene reference space.

We need another conversion mechanism to handle this discrepancy

<rdar://problem/16888499>

[chris fleizach]

Created attachment 231907 [details]
patch

[WebKit Commit Bot]

Attachment 231907 [details] did not pass style-queue:


ERROR: Source/WebKit2/UIProcess/ios/PageClientImplIOS.mm:338:  When wrapping a line, only indent 4 spaces.  [whitespace/indent] [3]
Total errors found: 1 in 23 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[chris fleizach]

Created attachment 231908 [details]
patch

[chris fleizach]

Created attachment 231922 [details]
patch

[Sam Weinig]

Is there a way we can do this without adding a new sync message? Can the UIProcess push the data needed for the transform down?

[chris fleizach]

Created attachment 232003 [details]
patch

[Simon Fraser (smfr)]

Comment on attachment 232003 [details]
patch

View in context: https://bugs.webkit.org/attachment.cgi?id=232003&action=review

> Source/WebKit/mac/WebCoreSupport/WebChromeClient.mm:613
> +IntPoint WebChromeClient::accessibilityScreenToRootView(const IntPoint& p) const
> +{
> +    return p;
> +}
> +
> +IntRect WebChromeClient::rootViewToAccessibilityScreen(const IntRect& r) const
> +{
> +    return r;
> +}
> +#endif

p -> point and r -> rect please.

> Source/WebKit2/WebProcess/WebPage/WebPage.cpp:2312
> +IntPoint WebPage::accessibilityScreenToRootView(const IntPoint& point)
> +{
> +    IntPoint windowPoint;
> +    sendSync(Messages::WebPageProxy::AccessibilityScreenToRootView(point), Messages::WebPageProxy::AccessibilityScreenToRootView::Reply(windowPoint));
> +    return windowPoint;
> +}
>  
> +IntRect WebPage::rootViewToAccessibilityScreen(const IntRect& rect)
> +{
> +    IntRect screenRect;
> +    sendSync(Messages::WebPageProxy::RootViewToAccessibilityScreen(rect), Messages::WebPageProxy::RootViewToAccessibilityScreen::Reply(screenRect));
> +    return screenRect;
> +}
> +#endif

Do we really have to use sync messages here? Can we push the to-screen mapping into the UI process?

[Anders Carlsson]

Comment on attachment 232003 [details]
patch

Would it be possible to keep a conversion matrix in the web process to avoid having to send sync messages?

[chris fleizach]

(In reply to comment #7)
> (From update of attachment 232003 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=232003&action=review
> 
> > Source/WebKit/mac/WebCoreSupport/WebChromeClient.mm:613
> > +IntPoint WebChromeClient::accessibilityScreenToRootView(const IntPoint& p) const
> > +{
> > +    return p;
> > +}
> > +
> > +IntRect WebChromeClient::rootViewToAccessibilityScreen(const IntRect& r) const
> > +{
> > +    return r;
> > +}
> > +#endif
> 
> p -> point and r -> rect please.
> 
> > Source/WebKit2/WebProcess/WebPage/WebPage.cpp:2312
> > +IntPoint WebPage::accessibilityScreenToRootView(const IntPoint& point)
> > +{
> > +    IntPoint windowPoint;
> > +    sendSync(Messages::WebPageProxy::AccessibilityScreenToRootView(point), Messages::WebPageProxy::AccessibilityScreenToRootView::Reply(windowPoint));
> > +    return windowPoint;
> > +}
> >  
> > +IntRect WebPage::rootViewToAccessibilityScreen(const IntRect& rect)
> > +{
> > +    IntRect screenRect;
> > +    sendSync(Messages::WebPageProxy::RootViewToAccessibilityScreen(rect), Messages::WebPageProxy::RootViewToAccessibilityScreen::Reply(screenRect));
> > +    return screenRect;
> > +}
> > +#endif
> 
> Do we really have to use sync messages here? Can we push the to-screen mapping into the UI process?

This is gonna be a bit tough. VoiceOver will message a WebAXObjectWrapper for it's frame, for example, (communicating directly to the WebProcess) and it's expecting an answer to be returned immediately.

for example the rootToScreenView() is also synchronous (which is why it's usable for AX code)

[chris fleizach]

(In reply to comment #8)
> (From update of attachment 232003 [details])
> Would it be possible to keep a conversion matrix in the web process to avoid having to send sync messages?

The conversion code is not simple. It's spread across AX code and UIKit and accounts for a number of conversions. I can email you offline what needs to happen

[chris fleizach]

Thanks Sam!

[WebKit Commit Bot]

Comment on attachment 232003 [details]
patch

Clearing flags on attachment: 232003

Committed r169310: <http://trac.webkit.org/changeset/169310>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.