Bug 273682 – [GTK] A few accessibility API tests are always failing in the bots

• Status: NEW
• Component: WebKitGTK

[Claudio Saavedra]

The following API tests are pretty much always failing in the API-GTK EWS bots:

/WebKitGTK/TestWebKitAccessibility:/webkit/WebKitAccessibility/selection/menulist
/WebKitGTK/TestWebKitAccessibility:/webkit/WebKitAccessibility/accessible/event-listener
/WebKitGTK/TestWebKitAccessibility:/webkit/WebKitAccessibility/selection/listbox
/WebKitGTK/TestWebKitAccessibility:/webkit/WebKitAccessibility/value/basic

I'll mark the failure in the expectations file as they're just slowing down the tests right now.

[Claudio Saavedra]

It's worth mentioning that these bots are testing GTK3.

[Claudio Saavedra]

Gardening here https://github.com/WebKit/WebKit/pull/28096

[Claudio Saavedra]

I am marking more failures and failures, the a11y tests are degrading considerably:

--- a/Tools/TestWebKitAPI/glib/TestExpectations.json
+++ b/Tools/TestWebKitAPI/glib/TestExpectations.json
@@ -49,7 +49,7 @@
                 "expected": {"gtk": {"status": ["TIMEOUT", "PASS"], "bug": "webkit.org/b/256557"}}
             },
             "/webkit/WebKitAccessibility/text/state-changed": {
-                "expected": {"gtk": {"status": ["FAIL", "TIMEOUT", "PASS"], "bug": "webkit.org/b/256557"}}
+                "expected": {"gtk": {"status": ["FAIL"], "bug": "webkit.org/b/256557"}}
             },
             "/webkit/WebKitAccessibility/selection/menulist": {
                 "expected": {"gtk": {"status": ["FAIL"], "bug": "webkit.org/b/273682"}}
@@ -62,6 +62,21 @@
             },
             "/webkit/WebKitAccessibility/value/basic": {
                 "expected": {"gtk": {"status": ["FAIL"], "bug": "webkit.org/b/273682"}}
+            },
+            "/webkit/WebKitAccessibility/accessible/list-markers": {
+                "expected": {"gtk": {"status": ["FAIL"], "bug": "webkit.org/b/273682"}}
+            },
+            "/webkit/WebKitAccessibility/accessible/state": {
+                "expected": {"gtk": {"status": ["FAIL"], "bug": "webkit.org/b/273682"}}
+            },
+            "/webkit/WebKitAccessibility/accessible/ignored-objects": {
+                "expected": {"gtk": {"status": ["FAIL", "PASS"], "bug": "webkit.org/b/273682"}}
+            },
+            "/webkit/WebKitAccessibility/document/load-events": {
+                "expected": {"gtk": {"status": ["FAIL"], "bug": "webkit.org/b/273682"}}
+            },
+            "/webkit/WebKitAccessibility/hyperlink/basic": {
+                "expected": {"gtk": {"status": ["FAIL", "PASS"], "bug": "webkit.org/b/273682"}}
             }
         }
     },

[Carlos Alberto Lopez Perez]

WebKitAccessibility/accessible/event-listener is timing out since migrated to the new infra, see bug 299598