Bug 196505 – [ Mac ] Layout Test accessibility/mac/press-not-work-for-disabled-menu-list.html is a flaky failure

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Shawn Roberts]

The following layout test is flaky on Mac WK1, WK2 Release and Debug

accessibility/mac/press-not-work-for-disabled-menu-list.html

Probable cause:

Test was added in https://trac.webkit.org/changeset/241190/webkit and is a flaky failure on its entire available dashboard history for Mojave WK1, but through local testing I am able to get it to fail on Release and Debug WK2 as well, just not as consistently. 

Local testing for WK1 yields approximately 20 failures in 500 iterations. WK2 testing yields about 2 failures in 500 iterations. 

Reproduced with:

run-webkit-tests accessibility/mac/press-not-work-for-disabled-menu-list.html --iterations 500 -f -1

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=accessibility%2Fmac%2Fpress-not-work-for-disabled-menu-list.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/accessibility/mac/press-not-work-for-disabled-menu-list-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/accessibility/mac/press-not-work-for-disabled-menu-list-actual.txt
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: line 56: TypeError: undefined is not an object (evaluating 'menulist.addNotificationListener')
     
 This tests that menu lists that are disabled will not be triggered by AXPress actions.
 
@@ -7,7 +8,7 @@
 Got notification: AXPressDidFail
 Got notification: AXPressDidFail
 Got notification: AXPressDidFail
-PASS successfullyParsed is true
+FAIL successfullyParsed should be true (of type boolean). Was undefined (of type undefined).
 
 TEST COMPLETE

[Shawn Roberts]

Marked flaky in https://trac.webkit.org/changeset/243750/webkit while waiting for a fix.

[Radar WebKit Bug Importer]

<rdar://problem/49532620>

[Andres Gonzalez]

Created attachment 375447 [details]
Patch

[chris fleizach]

Comment on attachment 375447 [details]
Patch

waiting for green bots to cq

[WebKit Commit Bot]

Comment on attachment 375447 [details]
Patch

Clearing flags on attachment: 375447

Committed r248189: <https://trac.webkit.org/changeset/248189>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.