Bug 210716 – pointermove event sometimes has incorrect pointerType of 'mouse' on touch interactions

• Component: UI Events
• Status: RESOLVED FIXED

On my 2017 iPad Pro 9.7", when running inside a WKWebView, a 'pointermove' listener attached to 'window' will sometimes incorrectly have its 'pointerType' property set to 'mouse' (instead of 'touch') even when no (Bluetooth) mouse is connected. This is very problematic because we use this event and its 'pointerType' property to detect if an actual mouse is connected, which now won't work.

[Radar WebKit Bug Importer]

<rdar://problem/62084817>

[Tim Horton]

Is this a change from a prior version of iOS? What version are you seeing this on?

We are tracking one regression in this area (rdar://problem/60422815) caused by r251320, but that would be a behavior change in iOS 13.4

[Antoine Quint]

Can you provide a testcase that shows the problem? Being able to reproduce would help diagnosing the issue. Additionally, did this reproduce this with an iOS 13 version prior to iOS 13.4 or is this new behavior in iOS 13.4?

(In reply to Tim Horton from comment #2)
> Is this a change from a prior version of iOS? What version are you seeing
> this on?
> 
> We are tracking one regression in this area (rdar://problem/60422815) caused
> by r251320, but that would be a behavior change in iOS 13.4

Sorry, I can't tell if it's a change -- the version I'm testing it on is 13.4.

Maybe a little background: We've developed a large WKWebView-based hybrid app that is a few years old and still based exclusively on touch events (touchstart, touchmove etc.).

When iOS 13.4 was introduced with "proper" mouse support, users started complaining about the mouse not doing anything, so we added a "mouse emulation layer" that generates synthetic touch events for received mouse events.

However, we want to avoid launching that layer unless the user actually uses a mouse, because it obviously is a huge hack and introduces overhead.

It seems to be very hard to properly detect if there is an actual mouse connected. We've tried various means, but the "PointerEvent.pointerType" route seemed the best and most logical, until we discovered that it is not reliable.

(In reply to Antoine Quint from comment #3)
> Can you provide a testcase that shows the problem? Being able to reproduce
> would help diagnosing the issue. Additionally, did this reproduce this with
> an iOS 13 version prior to iOS 13.4 or is this new behavior in iOS 13.4?

Unfortunately I can't tell if this behavior is new, because I only had to rely on the pointermove event since the new mouse support was introduced in iOS 13.4. (see my previous reply to Tim Horton).

What I can say is that it is quite rare... maybe 1 in 30 or 50 events has an incorrect pointerType (but it still is a crucial problem for our use case, again see other reply). It *might* occur more often when the swipes occur near the screen edges, but I'm not sure. Maybe it has to do with the global handling of the system gestures that can be invoked from there?

I could try to construct a test case, but could take me a few days.

Thanks for the quick attention to this issue!

(In reply to Tim Horton from comment #2)
> Is this a change from a prior version of iOS? What version are you seeing
> this on?
> 
> We are tracking one regression in this area (rdar://problem/60422815) caused
> by r251320, but that would be a behavior change in iOS 13.4

Just for completeness sake: the wrong pointerType definitely occurs BEFORE our "emulation layer" is launched, so it is not one of our synthetic mouse events (and our layer generates TouchEvents anyway, not PointerEvents.)

[Antoine Quint]

A test case would go a long way towards us fixing this. As it stands this bug report is not really actionable. Thanks for filing the issue.

OK, I've managed to isolate a test case. This took me literally hours and the bug is extremely odd:

It is actually connected to the HTML BUTTON element!

http://seven.systems/webkit-pointermove-test.html

The page has a global (window) 'pointermove' handler registered, which correctly gives 'touch' as pointerType.

HOWEVER, if you tap the BUTTON in the lower right corner, the global pointermove event will be fired (oddity #1), and it will have a pointerType of 'mouse'! (oddity #2).

I'm not ENTIRELY sure that this is exactly the same bug as the one I encountered in the larger app, but it WAS connected to various button elements in that app as well, so I assume it's the same or closely related.

You might want to change the title then!

[Antoine Quint]

This happens under this code path:

#0	0x00000004760272a8 in WebCore::dispatchPointerEventIfNeeded(WebCore::Element&, WebCore::MouseEvent const&, WebCore::PlatformMouseEvent const&, bool&) at /OpenSource/Source/WebCore/dom/Element.cpp:333
#1	0x0000000476026bf8 in WebCore::Element::dispatchMouseEvent(WebCore::PlatformMouseEvent const&, WTF::AtomString const&, int, WebCore::Element*) at /OpenSource/Source/WebCore/dom/Element.cpp:364
#2	0x0000000476bf48ff in WebCore::EventHandler::dispatchMouseEvent(WTF::AtomString const&, WebCore::Node*, int, WebCore::PlatformMouseEvent const&, WebCore::EventHandler::FireMouseOverOut) at /OpenSource/Source/WebCore/page/EventHandler.cpp:2661
#3	0x0000000476bf5dc6 in WebCore::EventHandler::handleMouseMoveEvent(WebCore::PlatformMouseEvent const&, WebCore::HitTestResult*, bool) at /OpenSource/Source/WebCore/page/EventHandler.cpp:2059
#4	0x0000000476bf553c in WebCore::EventHandler::mouseMoved(WebCore::PlatformMouseEvent const&) at /OpenSource/Source/WebCore/page/EventHandler.cpp:1919
#5	0x0000000474bf08fd in WebCore::EventHandler::dispatchSyntheticMouseMove(WebCore::PlatformMouseEvent const&) at /OpenSource/Source/WebCore/page/ios/EventHandlerIOS.mm:533
#6	0x000000010d960d5d in WebKit::dispatchSyntheticMouseMove(WebCore::Frame&, WebCore::FloatPoint const&, WTF::OptionSet<WebKit::WebEvent::Modifier>, int) at /OpenSource/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm:663
#7	0x000000010d95fe85 in WebKit::WebPage::handleSyntheticClick(WebCore::Node&, WebCore::FloatPoint const&, WTF::OptionSet<WebKit::WebEvent::Modifier>, int) at /OpenSource/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm:725
#8	0x000000010d9683c8 in WebKit::WebPage::commitPotentialTap(WTF::OptionSet<WebKit::WebEvent::Modifier>, WebKit::MonotonicObjectIdentifier<WebKit::TransactionIDType>, int) at /OpenSource/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm:1176

The MouseEvent has m_syntheticClickType = NoTap, perhaps we ought not create a synthetic pointer event for this type? Or maybe simply not dispatch this event on iOS?

[Antoine Quint]

Created attachment 398632 [details]
Patch

[EWS]

Committed r261246: <https://trac.webkit.org/changeset/261246>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 398632 [details].