Bug 262595 – REGRESSION(Sonoma): [ Sonoma ] 2 lookup menu-related tests are constant text failures.

• Status: NEW
• Component: Tools / Tests

[Ben Schwartz]

The following tests are constantly failing with text diffs on macOS Sonoma:

editing/mac/dictionary-lookup/dictionary-lookup.html
editing/selection/context-menu-text-selection-lookup.html

HISTORY:

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=editing%2Fmac%2Fdictionary-lookup%2Fdictionary-lookup.html&test=editing%2Fselection%2Fcontext-menu-text-selection-lookup.html

TEXT DIFFs:


 Some text to look up.
 
 Lookup string for normal text: 'look'.
-Lookup string for title-case phrase: 'New York'.
+Lookup string for title-case phrase: 'New'.
 Lookup string for unselectable text: 'null'.
_____

 This test checks that triggering the context menu selects/doesn't select using surround text as context (as platform-appropriate).
 
 New York
+FAILURE: We expected New York, but we got New.
 SUCCESS
 SUCCESS
-SUCCESS


DIFF URLs:

https://build.webkit.org/results/Apple-Sonoma-Release-WK1-Tests/268810@main%20(91)/editing/mac/dictionary-lookup/dictionary-lookup-pretty-diff.html
https://build.webkit.org/results/Apple-Sonoma-Release-WK1-Tests/268810@main%20(91)/editing/selection/context-menu-text-selection-lookup-pretty-diff.html

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Sonoma (14.0, 23A344/arm64) ToT using the following command:

run-webkit-tests --verbose --clobber-old-results editing/mac/dictionary-lookup/dictionary-lookup.html editing/selection/context-menu-text-selection-lookup.html

REGRESSION:

This test regressed with the bring-up of Sonoma testers (see bug #262489).

NOTES:

This seems to be an issue with how tests on Sonoma are run when the menus are open, which could be an OS regression.

[Radar WebKit Bug Importer]

<rdar://problem/116437531>

[EWS]

Test gardening commit 268820@main (f399a5ee439d): <https://commits.webkit.org/268820@main>

Reviewed commits have been landed. Closing PR #18602 and removing active labels.

[EWS]

Test gardening commit 280590@main (e604d0614fb7): <https://commits.webkit.org/280590@main>

Reviewed commits have been landed. Closing PR #30403 and removing active labels.

[Robert Jenner]

(In reply to EWS from comment #3)
> Test gardening commit 280590@main (e604d0614fb7):
> <https://commits.webkit.org/280590@main>
> 
> Reviewed commits have been landed. Closing PR #30403 and removing active
> labels.

I modified the existing expectations because these tests haven't been fixed, and it appears to impact both Sonoma and Sequoia.