Bug 230968 - Cleaning up expectations for iOS 15
Summary: Cleaning up expectations for iOS 15
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: iPhone / iPad Unspecified
: P2 Normal
Assignee: ayumi_kojima
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-29 10:38 PDT by ayumi_kojima
Modified: 2021-10-28 16:44 PDT (History)
4 users (show)

See Also:


Attachments
Update Expectations (4.10 KB, patch)
2021-09-29 11:20 PDT, ayumi_kojima
no flags Details | Formatted Diff | Diff
Update Expectations (4.10 KB, patch)
2021-09-29 11:22 PDT, ayumi_kojima
no flags Details | Formatted Diff | Diff
Update Expectations (5.97 KB, patch)
2021-09-29 14:09 PDT, ayumi_kojima
tsavell: review+
ews-feeder: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-09-29 10:38:29 PDT
fast/form/ios tests are currently marked as [ Pass Failure Timeout Crash ] in iOS-14.

fast/forms/ios/zoom-after-input-tap.html
fast/forms/ios/zoom-after-input-tap-wide-input.html
fast/forms/ios/focus-long-textarea.html
fast/forms/ios/focus-input-via-button.html
fast/forms/ios/focus-input-via-button-no-scaling.html
fast/forms/ios/focus-input-in-iframe.html
fast/forms/ios/focus-input-in-fixed.html
fast/forms/ios/accessory-bar-navigation.html

History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=fast%2Fforms%2Fios%2Fzoom-after-input-tap.html&test=fast%2Fforms%2Fios%2Fzoom-after-input-tap-wide-input.html&test=fast%2Fforms%2Fios%2Finputmode-none-with-hardware-keyboard.html&test=fast%2Fforms%2Fios%2Ffocus-long-textarea.html&test=fast%2Fforms%2Fios%2Ffocus-input-via-button.html&test=fast%2Fforms%2Fios%2Ffocus-input-via-button-no-scaling.html&test=fast%2Fforms%2Fios%2Ffocus-input-in-iframe.html&test=fast%2Fforms%2Fios%2Ffocus-input-in-fixed.html&test=fast%2Fforms%2Fios%2Faccessory-bar-navigation.html

The tests are initially marked as [ Pass Failure ] because they are intended to “fail or are flaky in non-internal iOS 14 simulator” https://trac.webkit.org/changeset/281749/webkit

Later [ Timeout ] and [ Crash ] are added here https://trac.webkit.org/changeset/281858/webkit 

Apparently constant timeout seen on iPad is because iPads don't zoom when form elements are focused.

The flaky crashes are not real crashes because of broken pipes (Bug 230513). 

Also, the constant failure and its diff indicate that the tests need rebaselining. 

Diff(iOS 14):

--- /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/fast/forms/ios/focus-input-in-fixed-expected.txt
+++ /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/fast/forms/ios/focus-input-in-fixed-actual.txt
@@ -1,4 +1,4 @@
 
 tap location	{ x: 40.000, y: 130.000 }
 scale	1.455
-visibleRect	{ left: 0.000, top: 44.683, width: 219.979, height: 376.713 }
+visibleRect	{ left: 0.000, top: 0.000, width: 219.979, height: 376.713 }
Comment 1 Radar WebKit Bug Importer 2021-09-29 10:38:53 PDT
<rdar://problem/83677926>
Comment 2 ayumi_kojima 2021-09-29 10:59:15 PDT
The numbers in the diffs found on the runs on iOS 14 and iOS 15 are the exact same, and local run on iOS 15 sim shows a different diff, I will hold off on rebaselining for now.

Diff (iOS 15)
--- /Volumes/Data/Builds/Star/ios/debugToTest/layout-test-results/fast/forms/ios/zoom-after-input-tap-expected.txt
+++ /Volumes/Data/Builds/Star/ios/debugToTest/layout-test-results/fast/forms/ios/zoom-after-input-tap-actual.txt
@@ -3,4 +3,4 @@
 
 tap location	{ x: 38.000, y: 862.000 }
 scale	1.455
-visibleRect	{ left: 0.000, top: 776.799, width: 219.979, height: 376.713 }
+visibleRect	{ left: 0.000, top: 655.820, width: 257.791, height: 444.775 }
Comment 3 ayumi_kojima 2021-09-29 11:20:56 PDT
Created attachment 439630 [details]
Update Expectations
Comment 4 ayumi_kojima 2021-09-29 11:22:46 PDT
Created attachment 439631 [details]
Update Expectations
Comment 5 ayumi_kojima 2021-09-29 14:09:36 PDT
Created attachment 439654 [details]
Update Expectations
Comment 6 ayumi_kojima 2021-09-29 15:16:14 PDT
Created bugs to track some fast tests that are marked in iOS-14:

fast/events/ios/viewport-shrink-to-fit-allows-double-tap.html (Bug 230987)
fast/selectors/matches-backtracking.html (Bug 230988)
fast/selectors/is-backtracking.html (Bug 230988)
Comment 7 EWS 2021-10-01 16:39:44 PDT
tsavell@apple.com does not have reviewer permissions according to https://raw.githubusercontent.com/WebKit/WebKit/main/metadata/contributors.json.

Rejecting attachment 439654 [details] from commit queue.
Comment 8 ayumi_kojima 2021-10-04 09:42:44 PDT
Updated expectations after Truitt's approval. Also, confirmed those tests need rebaselining on iOS 15. Added new expected files in iOS-15 folders: https://trac.webkit.org/changeset/283500/webkit
Comment 9 ayumi_kojima 2021-10-04 15:51:49 PDT
Rebaselined fast/events/ios/keydown-keyup-keypress-keys-in-non-editable-using-chinese-keyboard.html: https://trac.webkit.org/changeset/283520/webkit
Comment 10 ayumi_kojima 2021-10-05 10:40:25 PDT
https://trac.webkit.org/changeset/283523/webkit
Comment 11 ayumi_kojima 2021-10-07 14:02:01 PDT
Marked to skip http/tests/websocket/tests/hybi/bad-handshake-crash.html https://trac.webkit.org/changeset/283741/webkit