Bug 181806 - AX: roles-computedRoleString.html layout test should support enabling/disabling individual test cases
Summary: AX: roles-computedRoleString.html layout test should support enabling/disabli...
Alias: None
Product: WebKit
Classification: Unclassified
Component: Accessibility (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Normal
Assignee: Joanmarie Diggs
Keywords: InRadar
Depends on:
Reported: 2018-01-18 11:56 PST by Joanmarie Diggs
Modified: 2018-01-18 13:03 PST (History)
9 users (show)

See Also:

Patch (40.60 KB, patch)
2018-01-18 12:00 PST, Joanmarie Diggs
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joanmarie Diggs 2018-01-18 11:56:47 PST
The roles-computedRoleString.html layout test contains a large number of test cases, but no way to enable/disable an individual test case. Thus when there's a regression in just one specific element, gardeners wind up adding the test as a complete failure to TestExpectations and thus none of those tests get run on the affected platform. See, for example, bug 163383.

Given the importance of this particular layout test, and a lack of developer resources, we need a better solution. What is already being done for roles-exposed.html, namely adding a "data-platform" attribute to each element, would make it possible to enable/disable individual test cases for a given platform while allowing that platform to benefit from the remainder of the test.
Comment 1 Radar WebKit Bug Importer 2018-01-18 11:57:04 PST
Comment 2 Joanmarie Diggs 2018-01-18 12:00:48 PST
Created attachment 331646 [details]
Comment 3 Joanmarie Diggs 2018-01-18 12:01:45 PST
Chris: Any objections?
Comment 4 WebKit Commit Bot 2018-01-18 12:39:17 PST
Comment on attachment 331646 [details]

Clearing flags on attachment: 331646

Committed r227158: <https://trac.webkit.org/changeset/227158>
Comment 5 WebKit Commit Bot 2018-01-18 12:39:19 PST
All reviewed patches have been landed.  Closing bug.
Comment 6 Alexey Proskuryakov 2018-01-18 13:03:10 PST
Another approach would be to split the test into multiple files, and use the normal TestExpectations mechanism.