218891 2020-11-13 01:31:03 -0800 element.rect doesn't yield correct co-ordinates on iOS 2021-04-26 09:22:30 -0700 1 1 1 Unclassified WebKit WebDriver Other Unspecified Unspecified NEW InRadar P2 Normal --- 1 lukehill_uk nvasilyev bburg darin simon.fraser smoley webkit-bug-importer zalan oldest_to_newest 1707153 0 lukehill_uk 2020-11-13 01:31:03 -0800 I couldn't find the right location now to submit these. After initially submitting on selenium, then on appium, I'm now here. `element.rect` does not yield correct results on iOS compared to other browsers For full repro see: https://github.com/SeleniumHQ/selenium/issues/8869 / https://github.com/appium/ruby_lib_core/issues/285 ``` Bad candidates iPhone 8 - iOS 12 - #<struct Selenium::WebDriver::Rectangle x=17, **y=1**, width=293, height=48> iPhone 8 - iOS 13 - #<struct Selenium::WebDriver::Rectangle x=17, **y=1**, width=293, height=48> iPhone 11 - iOS 13 - #<struct Selenium::WebDriver::Rectangle x=17, **y=1**, width=332, height=48> Good candidates Windows 7 - Chrome v83 - #<struct Selenium::WebDriver::Rectangle x=17, y=**1032**, width=750, height=48> Windows 10 - Chrome v86 - #<struct Selenium::WebDriver::Rectangle x=17, y=**1060**, width=750, height=48> Windows 10 - Firefox v80 - #<struct Selenium::WebDriver::Rectangle x=17.0, y=**1010.0**, width=750.0, height=48.0> OSX Mojave - Safari 12 - #<struct Selenium::WebDriver::Rectangle x=17, y=**820**, width=750, height=48> ``` 1707723 1 lukehill_uk 2020-11-16 01:50:47 -0800 NB: During some parts of the testing. It's even possible to get the y ordinate to be negative in iOS. So something is definitely going wrong here!! 1708068 2 webkit-bug-importer 2020-11-16 18:23:48 -0800 <rdar://problem/71468283> 1708122 3 bburg 2020-11-16 22:18:41 -0800 Hello Luke, thanks for writing. It's not quite clear to me how you are scrolling. Could you please attach a test case so that there is no ambiguity? In general, iOS coordinates are computed and returned in visual viewport space, but the Get Element Rect command explicitly wants to add the scrollY to this coordinate system. In this case the scrollY of ~1000 is missing, so this indeed looks like a driver bug. 1708126 4 bburg 2020-11-16 22:23:18 -0800 From reading the code, it looks like we need to change WebAutomationSessionProxy::computeElementLayout() for the CoordinateSystem::Page case, so that it shifts the element bounds by window.scrollX and window.scrollY. Spec language is here: https://w3c.github.io/webdriver/#dfn-calculate-the-absolute-position 1708187 5 lukehill_uk 2020-11-17 04:08:23 -0800 I do use a JS "hack" to scroll. Which perhaps isn't computing well with the iOS Driver. But it works (i.e. it does scroll). For ref I do a window.scrollTo(10000,0) call. I "know" for iOS I could maybe do something different, but we don't care how the user scrolls - As for that webpage, we want to test it "stickies" the OISC Warning. To me this was the "best" way to test the sticky behaviour. I.e. the rect should be moved from its starting position. So my test grabs the starting y value, scrolls down, asserts it has changed. But for iOS it goes all weird. It works fine on Mojave 12 Safari so it suggests it's not globally broken with the safaridriver. So I'm not sure. 1753785 6 426962 bburg 2021-04-23 16:19:38 -0700 Created attachment 426962 [WIP] Patch v1.0 1754026 7 426962 darin 2021-04-25 10:47:06 -0700 Comment on attachment 426962 [WIP] Patch v1.0 View in context: https://bugs.webkit.org/attachment.cgi?id=426962&action=review > Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.cpp:662 > + // XXX XXX? 1754217 8 426962 simon.fraser 2021-04-26 09:22:30 -0700 Comment on attachment 426962 [WIP] Patch v1.0 View in context: https://bugs.webkit.org/attachment.cgi?id=426962&action=review > Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.cpp:668 > + scrollOffset.scale(2); // convertRectFromFrameClientToRootView subtracts scrollX/Y and we need to add it back and then again so it's positive This is very weird. 426962 2021-04-23 16:19:38 -0700 2021-04-23 16:20:08 -0700 [WIP] Patch v1.0 bug-218891-20210423161937.patch text/plain 4307 bburg U3VidmVyc2lvbiBSZXZpc2lvbjogMjc2NTE1CmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViQ29yZS9D aGFuZ2VMb2cgYi9Tb3VyY2UvV2ViQ29yZS9DaGFuZ2VMb2cKaW5kZXggZWNlMDZkYWI2OGEzYjAz NjEzNDkzMzJlOGZkMDFiMDExNDQ2MjdmNS4uMTA2NDQ1ZWU5ZjM0OTc5OTliODI5ZjdkOWVkNzg2 ZjZjYzFlMTU3YiAxMDA2NDQKLS0tIGEvU291cmNlL1dlYkNvcmUvQ2hhbmdlTG9nCisrKyBiL1Nv dXJjZS9XZWJDb3JlL0NoYW5nZUxvZwpAQCAtMSwzICsxLDEzIEBACisyMDIxLTA0LTIzICBCSiBC dXJnICA8YmJ1cmdAYXBwbGUuY29tPgorCisgICAgICAgIGVsZW1lbnQucmVjdCBkb2Vzbid0IHlp ZWxkIGNvcnJlY3QgY28tb3JkaW5hdGVzIG9uIGlPUworICAgICAgICBodHRwczovL2J1Z3Mud2Vi a2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjE4ODkxCisgICAgICAgIDxyZGFyOi8vcHJvYmxlbS83 MTQ2ODI4Mz4KKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAg ICAqIHBhZ2UvRnJhbWVWaWV3Lmg6IEV4cG9ydCBhYnNvbHV0ZVRvQ2xpZW50UmVjdCBmb3IgV2Vi S2l0LgorCiAyMDIxLTA0LTIzICBNaWNoYWVsIENhdGFuemFybyAgPG1jYXRhbnphcm9AaWdhbGlh LmNvbT4KIAogICAgICAgICBSZW1vdmUgdmlydHVhbCBmdW5jdGlvbiBjYWxscyBpbiBHcmFwaGlj c0xheWVyIGRlc3RydWN0b3JzCmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViS2l0L0NoYW5nZUxvZyBi L1NvdXJjZS9XZWJLaXQvQ2hhbmdlTG9nCmluZGV4IDMxYTk1NjQ1NTYyMWZiYzA3MDgyM2E0MzBk NDZmZDFhNTE4ZTg2NWUuLmE3MTU4OGJiY2Y5MDlmZjc1NTliMTlkMmFlZDlhM2JmNGVjYTk5ZDUg MTAwNjQ0Ci0tLSBhL1NvdXJjZS9XZWJLaXQvQ2hhbmdlTG9nCisrKyBiL1NvdXJjZS9XZWJLaXQv Q2hhbmdlTG9nCkBAIC0xLDMgKzEsMjAgQEAKKzIwMjEtMDQtMjMgIEJKIEJ1cmcgIDxiYnVyZ0Bh cHBsZS5jb20+CisKKyAgICAgICAgZWxlbWVudC5yZWN0IGRvZXNuJ3QgeWllbGQgY29ycmVjdCBj by1vcmRpbmF0ZXMgb24gaU9TCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3df YnVnLmNnaT9pZD0yMTg4OTEKKyAgICAgICAgPHJkYXI6Ly9wcm9ibGVtLzcxNDY4MjgzPgorCisg ICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFdJUDogYWRkIHNj cm9sbFgvWSB0byB0aGUgcmVwb3J0ZWQgZWxlbWVudCBib3VuZHMgcGVyIHNwZWMuCisgICAgICAg IFRPRE86CisgICAgICAgICAgICAtIGNsZWFuIHVwIC8gaW1wcm92ZSBjb21tZW50cworICAgICAg ICAgICAgLSBTdGlsbCBuZWVkcyBXUFQgdGVzdCBjYXNlcworICAgICAgICAgICAgLSB0ZXN0IHVz aW5nIFRvVCBpT1MgV2ViS2l0IGJ1aWxkCisKKyAgICAgICAgKiBXZWJQcm9jZXNzL0F1dG9tYXRp b24vV2ViQXV0b21hdGlvblNlc3Npb25Qcm94eS5jcHA6CisgICAgICAgIChXZWJLaXQ6OldlYkF1 dG9tYXRpb25TZXNzaW9uUHJveHk6OmNvbXB1dGVFbGVtZW50TGF5b3V0KToKKwogMjAyMS0wNC0y MyAgQnJlbnQgRnVsZ2hhbSAgPGJmdWxnaGFtQGFwcGxlLmNvbT4KIAogICAgICAgICBbaU9TXSBH UFUgUHJvY2VzcyBzYW5kYm94IGxhY2tzIElPTW9iaWxlRnJhbWVidWZmZXJVc2VyQ2xpZW50IG1l dGhvZCBmaWx0ZXIKZGlmZiAtLWdpdCBhL1NvdXJjZS9XZWJDb3JlL3BhZ2UvRnJhbWVWaWV3Lmgg Yi9Tb3VyY2UvV2ViQ29yZS9wYWdlL0ZyYW1lVmlldy5oCmluZGV4IGY0YTYwNzhlOTI2N2FjNDNm YTQzMWY1ZjllYzg2N2U5OTIxMmUwNTYuLjU3OWJiOTAzNTFlMGQyNDZkZGEwYzU1NmVjMTYzZmFi ZjRiNzQ4ODEgMTAwNjQ0Ci0tLSBhL1NvdXJjZS9XZWJDb3JlL3BhZ2UvRnJhbWVWaWV3LmgKKysr IGIvU291cmNlL1dlYkNvcmUvcGFnZS9GcmFtZVZpZXcuaApAQCAtNDk4LDcgKzQ5OCw3IEBAIHB1 YmxpYzoKICAgICBXRUJDT1JFX0VYUE9SVCBGbG9hdFJlY3QgYWJzb2x1dGVUb0RvY3VtZW50UmVj dChGbG9hdFJlY3QsIE9wdGlvbmFsPGZsb2F0PiBlZmZlY3RpdmVab29tID0gV1RGOjpudWxsb3B0 KSBjb25zdDsKICAgICBXRUJDT1JFX0VYUE9SVCBGbG9hdFBvaW50IGFic29sdXRlVG9Eb2N1bWVu dFBvaW50KEZsb2F0UG9pbnQsIE9wdGlvbmFsPGZsb2F0PiBlZmZlY3RpdmVab29tID0gV1RGOjpu dWxsb3B0KSBjb25zdDsKIAotICAgIEZsb2F0UmVjdCBhYnNvbHV0ZVRvQ2xpZW50UmVjdChGbG9h dFJlY3QsIE9wdGlvbmFsPGZsb2F0PiBlZmZlY3RpdmVab29tID0gV1RGOjpudWxsb3B0KSBjb25z dDsKKyAgICBXRUJDT1JFX0VYUE9SVCBGbG9hdFJlY3QgYWJzb2x1dGVUb0NsaWVudFJlY3QoRmxv YXRSZWN0LCBPcHRpb25hbDxmbG9hdD4gZWZmZWN0aXZlWm9vbSA9IFdURjo6bnVsbG9wdCkgY29u c3Q7CiAKICAgICBGbG9hdFNpemUgZG9jdW1lbnRUb0NsaWVudE9mZnNldCgpIGNvbnN0OwogICAg IFdFQkNPUkVfRVhQT1JUIEZsb2F0UmVjdCBkb2N1bWVudFRvQ2xpZW50UmVjdChGbG9hdFJlY3Qp IGNvbnN0OwpkaWZmIC0tZ2l0IGEvU291cmNlL1dlYktpdC9XZWJQcm9jZXNzL0F1dG9tYXRpb24v V2ViQXV0b21hdGlvblNlc3Npb25Qcm94eS5jcHAgYi9Tb3VyY2UvV2ViS2l0L1dlYlByb2Nlc3Mv QXV0b21hdGlvbi9XZWJBdXRvbWF0aW9uU2Vzc2lvblByb3h5LmNwcAppbmRleCBiMmVmYzhmZjhk M2IyM2YxOTE2NDFjNjFhOGYyMTNmMTBiMTNiMjdmLi4wODI5OGZlMjQ3YzhlYmEzMDNlN2Q0Yjlj NDdmNTJjYmQyOTZlYmM2IDEwMDY0NAotLS0gYS9Tb3VyY2UvV2ViS2l0L1dlYlByb2Nlc3MvQXV0 b21hdGlvbi9XZWJBdXRvbWF0aW9uU2Vzc2lvblByb3h5LmNwcAorKysgYi9Tb3VyY2UvV2ViS2l0 L1dlYlByb2Nlc3MvQXV0b21hdGlvbi9XZWJBdXRvbWF0aW9uU2Vzc2lvblByb3h5LmNwcApAQCAt NjU4LDkgKzY1OCwxNyBAQCB2b2lkIFdlYkF1dG9tYXRpb25TZXNzaW9uUHJveHk6OmNvbXB1dGVF bGVtZW50TGF5b3V0KFdlYkNvcmU6OlBhZ2VJZGVudGlmaWVyIHBhZwogICAgIGJvb2wgaXNPYnNj dXJlZCA9IGZhbHNlOwogCiAgICAgc3dpdGNoIChjb29yZGluYXRlU3lzdGVtKSB7Ci0gICAgY2Fz ZSBDb29yZGluYXRlU3lzdGVtOjpQYWdlOgotICAgICAgICByZXN1bHRFbGVtZW50Qm91bmRzID0g ZW5jbG9zaW5nSW50UmVjdChjb3JlRWxlbWVudC0+Ym91bmRpbmdDbGllbnRSZWN0KCkpOworICAg IGNhc2UgQ29vcmRpbmF0ZVN5c3RlbTo6UGFnZTogeworICAgICAgICAvLyBYWFgKKyAgICAgICAg Ly8gVGhpcyBpcyBpbiByb290IHZpZXcgY29vcmRpbmF0ZXMgd2hpY2ggYXJlIG5vdCAnY2xpZW50 JyBjb29yZGluYXRlcyBsaWtlIHdlIGV4cGVjdCBmcm9tIGdldEJvdW5kaW5nQ2xpZW50UmVjdHMu CisgICAgICAgIC8vIGNhbGwgIHJvb3RWaWV3VG9Db250ZW50cyAtLT4gYWJzb2x1dGVUb0NsaWVu dAorICAgICAgICBGbG9hdFJlY3QgZWxlbWVudEJvdW5kc0luUm9vdENvb3JkaW5hdGVzID0gZW5j bG9zaW5nSW50UmVjdChjb252ZXJ0UmVjdEZyb21GcmFtZUNsaWVudFRvUm9vdFZpZXcoZnJhbWVW aWV3LCBjb3JlRWxlbWVudC0+Ym91bmRpbmdDbGllbnRSZWN0KCkpKTsKKyAgICAgICAgcmVzdWx0 RWxlbWVudEJvdW5kcyA9IGVuY2xvc2luZ0ludFJlY3QobWFpblZpZXctPmFic29sdXRlVG9DbGll bnRSZWN0KGVsZW1lbnRCb3VuZHNJblJvb3RDb29yZGluYXRlcykpOworICAgICAgICBhdXRvIHNj cm9sbE9mZnNldCA9IG1haW5WaWV3LT5jb250ZW50c1Njcm9sbFBvc2l0aW9uKCk7CisgICAgICAg IHNjcm9sbE9mZnNldC5zY2FsZSgyKTsgLy8gY29udmVydFJlY3RGcm9tRnJhbWVDbGllbnRUb1Jv b3RWaWV3IHN1YnRyYWN0cyBzY3JvbGxYL1kgYW5kIHdlIG5lZWQgdG8gYWRkIGl0IGJhY2sgYW5k IHRoZW4gYWdhaW4gc28gaXQncyBwb3NpdGl2ZQorICAgICAgICByZXN1bHRFbGVtZW50Qm91bmRz Lm1vdmVCeShzY3JvbGxPZmZzZXQpOwogICAgICAgICBicmVhazsKKyAgICB9CiAgICAgY2FzZSBD b29yZGluYXRlU3lzdGVtOjpMYXlvdXRWaWV3cG9ydDogewogICAgICAgICBhdXRvIGVsZW1lbnRC b3VuZHNJblJvb3RDb29yZGluYXRlcyA9IGNvbnZlcnRSZWN0RnJvbUZyYW1lQ2xpZW50VG9Sb290 VmlldyhmcmFtZVZpZXcsIGNvcmVFbGVtZW50LT5ib3VuZGluZ0NsaWVudFJlY3QoKSk7CiAgICAg ICAgIHJlc3VsdEVsZW1lbnRCb3VuZHMgPSBlbmNsb3NpbmdJbnRSZWN0KG1haW5WaWV3LT5hYnNv bHV0ZVRvTGF5b3V0Vmlld3BvcnRSZWN0KG1haW5WaWV3LT5yb290Vmlld1RvQ29udGVudHMoZWxl bWVudEJvdW5kc0luUm9vdENvb3JkaW5hdGVzKSkpOwo=