Bug 222988 – [ BigSur wk2 arm64 ] quite a few fast/forms (Layout-Tests) are text failing on Apple Silicon

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Robert Jenner]

The tests below appear to be frequently text failing on BigSur wk2 Apple Silicon Release and Debug:

fast/forms/date/date-editable-components/date-editable-components-keyboard-events.html
fast/forms/date/date-editable-components/date-editable-components-mouse-events.html
fast/forms/date/date-editable-components/date-picker-update-on-edit.html
fast/forms/date/date-input-rendering-basic.html
fast/forms/date/date-pseudo-elements.html
fast/forms/datetimelocal/datetimelocal-editable-components/datetimelocal-editable-components-focus-and-blur-events.html
fast/forms/datetimelocal/datetimelocal-editable-components/datetimelocal-editable-components-keyboard-events.html
fast/forms/datetimelocal/datetimelocal-editable-components/datetimelocal-editable-components-mouse-events.html
fast/forms/datetimelocal/datetimelocal-editable-components/datetimelocal-editable-components-second-and-millisecond-field.html
fast/forms/datetimelocal/datetimelocal-editable-components/datetimelocal-picker-update-on-edit.html
fast/forms/time-input-svg-font.html
fast/forms/time/time-editable-components/time-editable-components-focus-and-blur-events.html
fast/forms/time/time-editable-components/time-editable-components-keyboard-events.html
fast/forms/time/time-editable-components/time-editable-components-mouse-events.html
fast/forms/time/time-editable-components/time-editable-components-second-and-millisecond-field.html
fast/forms/time/time-input-rendering-basic.html
fast/text/hyphenate-locale.html

HISTORY URL:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=fast%2Fforms%2Fdate%2Fdate-editable-components%2Fdate-editable-components-keyboard-events.html&test=fast%2Fforms%2Fdate%2Fdate-editable-components%2Fdate-editable-components-mouse-events.html&test=fast%2Fforms%2Fdate%2Fdate-editable-components%2Fdate-picker-update-on-edit.html&test=fast%2Fforms%2Fdate%2Fdate-input-rendering-basic.html&test=fast%2Fforms%2Fdate%2Fdate-pseudo-elements.html&test=fast%2Fforms%2Fdatetimelocal%2Fdatetimelocal-editable-components%2Fdatetimelocal-editable-components-focus-and-blur-events.html&test=fast%2Fforms%2Fdatetimelocal%2Fdatetimelocal-editable-components%2Fdatetimelocal-editable-components-keyboard-events.html&test=fast%2Fforms%2Fdatetimelocal%2Fdatetimelocal-editable-components%2Fdatetimelocal-editable-components-mouse-events.html&test=fast%2Fforms%2Fdatetimelocal%2Fdatetimelocal-editable-components%2Fdatetimelocal-editable-components-second-and-millisecond-field.html&test=fast%2Fforms%2Fdatetimelocal%2Fdatetimelocal-editable-components%2Fdatetimelocal-picker-update-on-edit.html&test=fast%2Fforms%2Ftime-input-svg-font.html&test=fast%2Fforms%2Ftime%2Ftime-editable-components%2Ftime-editable-components-focus-and-blur-events.html&test=fast%2Fforms%2Ftime%2Ftime-editable-components%2Ftime-editable-components-keyboard-events.html&test=fast%2Fforms%2Ftime%2Ftime-editable-components%2Ftime-editable-components-mouse-events.html&test=fast%2Fforms%2Ftime%2Ftime-editable-components%2Ftime-editable-components-second-and-millisecond-field.html&test=fast%2Fforms%2Ftime%2Ftime-input-rendering-basic.html&test=fast%2Ftext%2Fhyphenate-locale.html&architecture=arm64


All of tests have the same failure history. They passed the first few times since the test was introduced for Apple Silicon, but then started failing consistently.

[Radar WebKit Bug Importer]

<rdar://problem/75230416>

[Robert Jenner]

I do not have access to an Apple Silicon Mac, so I cannot reproduce the failures. 

However, I will say that they are the same test family, and they share the same failure history. These tests may just need an adjustment to run on Apple Silicon.

[Robert Jenner]

Created attachment 422750 [details]
File for all of the Text Diffs

These are all of the text DIFF files for all 17 of the failures. I put them in a folder, and also broke them down by the type of failure they were having. There are four categories of text failures these tests are having:

blur/FocusEventsFired
input.value
numberOfFields
RenderText

[Aditya Keerthi]

Based on the diffs attached it appears as though these tests are failing due to a different language/locale on the Apple Silicon bots.

The system language affects the order of fields for the date/time tests, leading to different values when editing the input.

Note that fast/text/hyphenate-locale.html doesn't belong to the same family as the other tests (date/time inputs), but is a test that is affected by locale.

The easiest way to fix this would be to update the system language/locale on the bots. A better solution would be to make these tests independent of the system language/locale. I will see if it's possible to rewrite these tests.

[Ryan Haddad]

I did find that a handful of these machines were configured with en-GB as the language, though I'm not sure how they ended up in this state.

I have now ensured that they are all configured as en-US / en_US, so lets continue to monitor the results to see what happens.

[Ryan Haddad]

The failures disappeared on the next run after I reset the language: https://build.webkit.org/results/Apple-BigSur-Release-AppleSilicon-WK2-Tests/r274171%20(114)/results.html

However, 8/10 of these bots are back on en-GB again, so the theory that this is due to a test changing the language and not returning it to the original one is back on the table

Presumably, https://bugs.webkit.org/show_bug.cgi?id=222619 would be a culprit.

[Chris Dumez]

Committed r274195 (235113@main): <https://commits.webkit.org/235113@main>

[Chris Dumez]

Please reset the language on these bots to en-US one more time. My patch should make sure it does not get modified to en-GB again.

[Alexey Proskuryakov]

Is there a root cause with potentially user facing symptoms to be worried about?

[Chris Dumez]

(In reply to Alexey Proskuryakov from comment #9)
> Is there a root cause with potentially user facing symptoms to be worried
> about?

Tracked by Bug 222825. It seems the languagechange event may not be firing on Big Sur / Apple Silicon (even though it does on Intel). This is likely not a regression, just a new find because of the newly added API test.