Bug 223291

Summary:	[ macOS wk2 ARM 64 ] 2 accessibility/mac/value-change/ layout-tests are flakey text failures
Product:	WebKit	Reporter:	Robert Jenner <jenner>
Component:	Accessibility	Assignee:	Nobody <webkit-unassigned>
Status:	NEW ---
Severity:	Normal	CC:	webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Robert Jenner 2021-03-16 17:25:08 PDT

accessibility/mac/value-change/value-change-user-info-textfield.html
accessibility/mac/value-change/value-change-user-info-textarea.html

are flakey text failures on macOS BigSur wk2 on Apple Silicon only. 

Both tests are flakey text failing on release, but "accessibility/mac/value-change/value-change-user-info-textarea.html" has text failed once or twice on debug. 

HISTORY URL:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=accessibility%2Fmac%2Fvalue-change%2Fvalue-change-user-info-textarea.html&test=accessibility%2Fmac%2Fvalue-change%2Fvalue-change-user-info-textfield.html&platform=mac&architecture=arm64

TEXT DIFF for "accessibility/mac/value-change/value-change-user-info-textfield.html":
 PASS actualChangeValues[resultIndex][1] is expectedValues[resultIndex][1]
 PASS actualEditTypes[resultIndex][1] is "Typing"
 PASS actualChangeTypes[resultIndex] is AXTextStateChangeTypeEdit
-PASS actualChangeValues[resultIndex][0] is expectedValues[resultIndex][0]
+FAIL actualChangeValues[resultIndex][0] should be 9. Was 8.
 PASS actualEditTypes[resultIndex][0] is "Delete"
-PASS actualChangeValues[resultIndex][1] is expectedValues[resultIndex][1]
+FAIL actualChangeValues[resultIndex][1] should be 8. Was 9.
 PASS actualEditTypes[resultIndex][1] is "Insert"
 PASS actualChangeTypes[resultIndex] is AXTextStateChangeTypeEdit
-PASS actualChangeValues[resultIndex][0] is expectedValues[resultIndex][0]
+FAIL actualChangeValues[resultIndex][0] should be 8. Was 9.
 PASS actualEditTypes[resultIndex][0] is "Delete"
 PASS actualChangeValues[resultIndex][1] is expectedValues[resultIndex][1]
 PASS actualEditTypes[resultIndex][1] is "Insert"
 PASS successfullyParsed is true
+Some tests failed.

TEXT DIFF for "accessibility/mac/value-change/value-change-user-info-textarea.html":
 PASS actualChangeValues[resultIndex][1] is expectedValues[resultIndex][1]
 PASS actualEditTypes[resultIndex][1] is "Typing"
 PASS actualChangeTypes[resultIndex] is AXTextStateChangeTypeEdit
-PASS actualChangeValues[resultIndex][0] is expectedValues[resultIndex][0]
+FAIL actualChangeValues[resultIndex][0] should be 6. Was 5.
 PASS actualEditTypes[resultIndex][0] is "Delete"
-PASS actualChangeValues[resultIndex][1] is expectedValues[resultIndex][1]
+FAIL actualChangeValues[resultIndex][1] should be 5. Was 6.
 PASS actualEditTypes[resultIndex][1] is "Insert"
 PASS actualChangeTypes[resultIndex] is AXTextStateChangeTypeEdit
-PASS actualChangeValues[resultIndex][0] is expectedValues[resultIndex][0]
+FAIL actualChangeValues[resultIndex][0] should be 5. Was 6.
 PASS actualEditTypes[resultIndex][0] is "Delete"
 PASS actualChangeValues[resultIndex][1] is expectedValues[resultIndex][1]
 PASS actualEditTypes[resultIndex][1] is "Insert"
 PASS successfullyParsed is true
+Some tests failed.

Comment 1 Radar WebKit Bug Importer 2021-03-16 17:30:47 PDT

<rdar://problem/75503005>

Comment 2 Robert Jenner 2021-03-16 17:38:18 PDT

Updated test expectations here to Pass Failure while test is reviewed:

https://trac.webkit.org/changeset/274540/webkit

Comment 3 Robert Jenner 2021-03-16 17:38:53 PDT

The failures appear to only occur on Apple Silicon Macs, and as such I cannot reproduce the failures because I do not have access to one.

Comment 4 Alexey Proskuryakov 2021-03-17 11:16:52 PDT

This seems unlikely to a be a truly Apple Silicon specific issue.