Bug 196505 - [ Mac ] Layout Test accessibility/mac/press-not-work-for-disabled-menu-list.html is a flaky failure
Summary: [ Mac ] Layout Test accessibility/mac/press-not-work-for-disabled-menu-list.h...
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2019-04-02 12:02 PDT by Shawn Roberts
Modified: 2019-08-02 16:20 PDT (History)
14 users (show)

See Also:

Patch (2.94 KB, patch)
2019-08-02 13:25 PDT, Andres Gonzalez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-04-02 12:02:42 PDT
The following layout test is flaky on Mac WK1, WK2 Release and Debug


Probable cause:

Test was added in https://trac.webkit.org/changeset/241190/webkit and is a flaky failure on its entire available dashboard history for Mojave WK1, but through local testing I am able to get it to fail on Release and Debug WK2 as well, just not as consistently. 

Local testing for WK1 yields approximately 20 failures in 500 iterations. WK2 testing yields about 2 failures in 500 iterations. 

Reproduced with:

run-webkit-tests accessibility/mac/press-not-work-for-disabled-menu-list.html --iterations 500 -f -1

Flakiness Dashboard:



--- /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/accessibility/mac/press-not-work-for-disabled-menu-list-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/accessibility/mac/press-not-work-for-disabled-menu-list-actual.txt
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: line 56: TypeError: undefined is not an object (evaluating 'menulist.addNotificationListener')
 This tests that menu lists that are disabled will not be triggered by AXPress actions.
@@ -7,7 +8,7 @@
 Got notification: AXPressDidFail
 Got notification: AXPressDidFail
 Got notification: AXPressDidFail
-PASS successfullyParsed is true
+FAIL successfullyParsed should be true (of type boolean). Was undefined (of type undefined).
Comment 1 Shawn Roberts 2019-04-02 12:10:02 PDT
Marked flaky in https://trac.webkit.org/changeset/243750/webkit while waiting for a fix.
Comment 2 Radar WebKit Bug Importer 2019-04-02 13:47:28 PDT
Comment 3 Andres Gonzalez 2019-08-02 13:25:31 PDT
Created attachment 375447 [details]
Comment 4 chris fleizach 2019-08-02 13:36:59 PDT
Comment on attachment 375447 [details]

waiting for green bots to cq
Comment 5 WebKit Commit Bot 2019-08-02 16:20:38 PDT
Comment on attachment 375447 [details]

Clearing flags on attachment: 375447

Committed r248189: <https://trac.webkit.org/changeset/248189>
Comment 6 WebKit Commit Bot 2019-08-02 16:20:39 PDT
All reviewed patches have been landed.  Closing bug.