Bug 200216 - [iPadOS] slides.google.com: tapping near cursor in a slide title focuses the speaker notes
Summary: [iPadOS] slides.google.com: tapping near cursor in a slide title focuses the ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: UI Events (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryosuke Niwa
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-07-28 22:48 PDT by Maciej Stachowiak
Modified: 2019-08-06 20:15 PDT (History)
6 users (show)

See Also:


Attachments
Fixes the bug (16.68 KB, patch)
2019-08-06 17:50 PDT, Ryosuke Niwa
no flags Details | Formatted Diff | Diff
EWS build fix (16.71 KB, patch)
2019-08-06 19:00 PDT, Ryosuke Niwa
wenson_hsieh: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Maciej Stachowiak 2019-07-28 22:48:44 PDT
Tapping the near cursor in a title in a slides.google.com slide focuses the speaker notes.

Steps to reproduce:
0) Install iOS 13 beta or newer on a non-mini iPad (no hardware keyboard attached)
1) Visit https://slides.google.com/ and log in with a Google account (if not already logged in)
2) Create a new presentation using the orange 'Your big idea' presentation template.
3) Go to the second slide (e.g. by tapping it in the sidebar).
3) Tap the title on the slide.
     (The cursor appears at this point, but, consistent with Google Docs and Google Sheets, it takes ab extra tap near the cursor to bring up the keyboard)
4) Tap near the cursor.

Results:
- Focus is placed in speaker notes instead of the title.
- The slide shrinks to really tiny.

It's possible these behaviors are connected somehow, both seem undesirable.
Comment 1 Radar WebKit Bug Importer 2019-07-28 22:49:13 PDT
<rdar://problem/53648131>
Comment 2 Maciej Stachowiak 2019-07-29 00:26:42 PDT
None of this weirdness happens with a physical keyboard attached, by the way.
Comment 3 Daniel Bates 2019-08-01 13:55:02 PDT
slides.google.com has seems to always have an <iframe> as their active element whose content document's <body> has inputMode = "none" even when you tap once. When you tap twice, slides remove's the inputMode HTML attribute entirely => unspecified input mode => default behavior and we show the keyboard.

As to why the site works when you have a hardware keyboard attached...because we do not respect inputMode when you have one attached 😀.

Haven't tried what happens on a Chromebook or Surface, but in absence of doing so, I'm leaning towards this bug being a site issue. Might be able to add a site quirk though for slides.google.com. Here's what we know towards a quirk:

1. The user explicitly tapped on something
2. The tapped thing causes a focusing/re-focusing of a content editable element (their active element <iframe>'s content document's <body> is content editable)
3. Slides does not appear to have custom keyboard UI.
Comment 4 Daniel Bates 2019-08-01 14:00:27 PDT
(In reply to Daniel Bates from comment #3)
> slides.google.com has seems to always have an <iframe> as their active
> element whose content document's <body> has inputMode = "none" even when you
> tap once. When you tap twice, slides remove's the inputMode HTML attribute
> entirely => unspecified input mode => default behavior and we show the
> keyboard.
> 
> As to why the site works when you have a hardware keyboard
> attached...because we do not respect inputMode when you have one attached 😀.
> 
> Haven't tried what happens on a Chromebook or Surface, but in absence of
> doing so, I'm leaning towards this bug being a site issue. Might be able to
> add a site quirk though for slides.google.com. Here's what we know towards a
> quirk:
> 
> 1. The user explicitly tapped on something
> 2. The tapped thing causes a focusing/re-focusing of a content editable
> element (their active element <iframe>'s content document's <body> is
> content editable)

Cannot use this as a factor because every tapped element (tapping a slide thumbnail etc etc) seems to refocus <body> and <body> is always content editable. That sucks.
Comment 5 Daniel Bates 2019-08-01 14:01:16 PDT
Over to Evangelism...
Comment 7 Daniel Bates 2019-08-02 11:16:48 PDT
Taking this bug back.... WebKit does not consider the fact that causing the keyboard to show or hide as a result of an HTML inputmode change can effect the position of touchevents (and mouse events) that are being processed. More specifically, the bug on slides.google.com occurs because the UI process is notified of the HTML inputMode change and we reload the input views between the touchdown and touchup as debug logging shows:

### -_webTouchEventsRecognized:; type: TouchStart
### -_webTouchEventsRecognized:; type: TouchEnd
### -_webTouchEventsRecognized:; type: TouchStart
### -_didUpdateInputMode:
### -_keyboardWillShow:
### -_webTouchEventsRecognized:; type: TouchEnd
### -_keyboardDidShow:

My guess is that Google changes focus on touchup (aka TouchEnd) and the showing of the keyboard effects the tapped point on the page.
Comment 8 Ryosuke Niwa 2019-08-06 17:50:13 PDT
Created attachment 375672 [details]
Fixes the bug
Comment 9 Ryosuke Niwa 2019-08-06 19:00:03 PDT
Created attachment 375674 [details]
EWS build fix
Comment 10 Wenson Hsieh 2019-08-06 19:08:42 PDT
Comment on attachment 375674 [details]
EWS build fix

View in context: https://bugs.webkit.org/attachment.cgi?id=375674&action=review

> Source/WebKit/ChangeLog:16
> +        touchend / pointerup evnets are dispatched but this is rather risky since that could affact random

Nit - evnets => events, affact => affect

> Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm:944
> +    if (m_pendingInputModeChange)
> +        pageClient().focusedElementDidChangeInputMode(*m_pendingInputModeChange);

Would it make sense to clear out m_pendingInputModeChange after we use it here?
Comment 11 Ryosuke Niwa 2019-08-06 20:12:12 PDT
(In reply to Wenson Hsieh from comment #10)
> Comment on attachment 375674 [details]
> EWS build fix
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=375674&action=review
> 
> > Source/WebKit/ChangeLog:16
> > +        touchend / pointerup evnets are dispatched but this is rather risky since that could affact random
> 
> Nit - evnets => events, affact => affect


> > Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm:944
> > +    if (m_pendingInputModeChange)
> > +        pageClient().focusedElementDidChangeInputMode(*m_pendingInputModeChange);
> 
> Would it make sense to clear out m_pendingInputModeChange after we use it
> here?

Yeah, we should surely do that.
Comment 12 Ryosuke Niwa 2019-08-06 20:15:21 PDT
Committed r248338: <https://trac.webkit.org/changeset/248338>