Bug 170981 – Pinch-zoom should be invisible to the application by default

• Component: DOM
• Status: NEW

[Rick Byers]

Developers typically expect APIs like MouseEvent.clientX/Y and getBoundingClientRect to use the same co-ordinate system as position:fixed (what we've been calling "layout viewport co-ordinates").  In Safari's new visual viewport model these APIs return visual co-ordinates instead, resulting in some site bugs under pinch-zoom.  As a simple example see https://rbyers.github.io/inputCoords.html - when pinch-zooming in the red box is wrong.

In addition, some sites don't expect scroll offsets to change when panning the pinch zoom viewport, or that changing the scroll position will reset the position of the pinch zoom viewport.  For example on apple.com if you zoom in and click the search button in the top bar the viewport doesn't get centered correctly on the search field because there's a window.scrollTo(0,0) call.

Here's some example sites that are broken in Safari today:
https://docs.google.com/document/d/1jxWEzwOT2XgSsg5k2a_3D7PR0bXnB4bL9rATW-Z6egY/edit

Talking with smfr@ he agrees with the principle that pinch-zoom should be completely transparent to the application by default.  Developers generally are not thinking about the impact of pinch-zoom, and pinch-zoom is primarily a UA/accessibility feature.  He says he'd like to change WebKit to make all these APIs use layout viewport co-ordinates instead of visual viewport ones (i.e. to match Chrome's behavior with the chrome://flags/#inert-visual-viewport flag enabled - see https://crbug.com/489206)

[Rick Byers]

See bug 170982 for a possible way to explicitly expose pinch-zoom to applications if desired.

[Rick Byers]

An old list of APIs and which co-ordinate systems they currently use in blink:
https://docs.google.com/spreadsheets/d/11DfDDa-s1hePVwBn3PZidlPJZ9ahhkJ44dyuMiOQtrc/edit#gid=0

We intend to change all the ones marked 'visual viewport' to 
'layout viewport'.

[Radar WebKit Bug Importer]

<rdar://problem/31702298>

[Simon Fraser (smfr)]

This seems like a reasonable approach.

[David Bokan]

Would it make sense to only make this change at the same time as shipping the viewport API (https://github.com/WICG/ViewportAPI)? There are some reasonable though niche use cases that lose some abilities when pinch-zoom is made invisible. We got some non-trivial push-back from web devs in http://crbug.com/571297 when we did this.

The presence of the API would also be a convenient way to feature-detect the change, which is a request we've gotten: https://github.com/WICG/ViewportAPI/issues/24. Otherwise, the page has no way to  know how window.innerWidth et al. will behave.

[Simon Fraser (smfr)]

I'm seeing issues with baidu.com, which is using jQuery's offset() function, and using the result to do a scrollTo().

offset().top is basically elementgetBoundingClientRect().top + window.pageYOffset, so using the result for scrollTo() doesn't scroll that element to the top if the top of the layout viewport is not at scrollY.

This implies that scroll positions should be the origin of the layout viewport rect?

[David Bokan]

Right, that's a common pattern I've seen is to assume that "position in document" = "getBoundingClientRect + window.scrollX/Y".

This doesn't work in Safari (and Chrome until M61) since getBoundingClientRect returns the position relative to the layout viewport but window.scrollX/Y is the position of the visual viewport. In M61, we've made window.scrollX/Y return the offset of the layout viewport.

Note: we've also made window.scrollTo scroll only the layout viewport.

[Simon Fraser (smfr)]

(In reply to David Bokan from comment #7)
> Right, that's a common pattern I've seen is to assume that "position in
> document" = "getBoundingClientRect + window.scrollX/Y".
> 
> This doesn't work in Safari (and Chrome until M61) since
> getBoundingClientRect returns the position relative to the layout viewport
> but window.scrollX/Y is the position of the visual viewport. In M61, we've
> made window.scrollX/Y return the offset of the layout viewport.
> 
> Note: we've also made window.scrollTo scroll only the layout viewport.

Thanks David. Could you update https://docs.google.com/spreadsheets/d/11DfDDa-s1hePVwBn3PZidlPJZ9ahhkJ44dyuMiOQtrc/edit#gid=0 with what Chrome is doing now?

Changing scrollX/Y to use the layout viewport is a larger change than I can do now. I wonder if we should get a jQuery fix? I think it would work to do something like element.getBoundingClientRct().y + documentElement.getBoundingClientRct() - window.pageYOffset?

[David Bokan]

> Thanks David. Could you update
> https://docs.google.com/spreadsheets/d/11DfDDa-
> s1hePVwBn3PZidlPJZ9ahhkJ44dyuMiOQtrc/edit#gid=0 with what Chrome is doing
> now?

Done. Chrome should be all-layout starting in M61 (I actually found a bug going through the list - Element.scrollTo and friends is still visual) but I should have that fixed in time).

> 
> Changing scrollX/Y to use the layout viewport is a larger change than I can
> do now. I wonder if we should get a jQuery fix? I think it would work to do
> something like element.getBoundingClientRct().y +
> documentElement.getBoundingClientRct() - window.pageYOffset?

Yeah, I would expect that should work

[Sommer Panage]

Still seeing this issue in Safari 16.2 on macOS 13.1 Beta (22C5044e). Is there any chance it could get re-hydrated? It's very problematic for our Quip customers who are using pinch-to-zoom.

[Simon Fraser (smfr)]

Can you be more specific about the problems this causes in Quip?

[Sommer Panage]

Created attachment 463565 [details]
Chrome pinch to zoom behaving well in Quip document

[Sommer Panage]

Created attachment 463566 [details]
Safari pinch to zoom behaving badly in Quip document

[Sommer Panage]

Added a couple videos to help - main symptom we're seeing is a big offset jump on initial zoom in and jitter on horizontal scroll.

[Simon Fraser (smfr)]

(In reply to Sommer Panage from comment #14)
> Added a couple videos to help - main symptom we're seeing is a big offset
> jump on initial zoom in and jitter on horizontal scroll.

What's happening here is that Quip is aggressively snapping the scroll position to 0,0 when it receives a scroll event. It's true that fixing this bug means that the scroll position will remain 0,0 on zooming, but Quip could also chill a little and not do this aggressive scrollTo(0,0).