Bug 265498
| Summary: | [update-test-expectations-from-bugzilla] analyze layout test results | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Sam Sneddon [:gsnedders] <gsnedders> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | webkit-bug-importer |
| Priority: | P2 | Keywords: | InRadar |
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=266533 | ||
Sam Sneddon [:gsnedders]
similar to the analyze-layout-tests-results step (c.f., https://github.com/WebKit/WebKit/blob/5965a5a92e802141a8ec48b1a83c9bb88069f1e1/Tools/CISupport/ews-build/steps.py#L4454), we should not *just* take the initial layout-tests run as the definitive set of results
this should vastly reduce the number of failed EWS runs as a result of running the tool, and avoid changing tests because they are ungardened on main.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Radar WebKit Bug Importer
<rdar://problem/118909142>
Sam Sneddon [:gsnedders]
https://github.com/WebKit/WebKit/pull/21838 is laying some of the group work there, and the comment I just posted there explains where I expect to build on top of that PR:
> The end goal is to move to having some class representing a group of runs, consisting of one or more layout tests runs plus zero or more runs without changes. We can then have that class tell us "what are the new (stable) failures" and use that.
Sam Sneddon [:gsnedders]
this also probably needs to either deal directly with the results database (which requires knowing the base revision), or pull the data from the EWS run, as we often ignore (and even avoid re-runs) test failures based on it: https://ews-build.webkit.org/#/builders/36/builds/27897