NEW 265498
[update-test-expectations-from-bugzilla] analyze layout test results 
https://bugs.webkit.org/show_bug.cgi?id=265498
Summary [update-test-expectations-from-bugzilla] analyze layout test results
Sam Sneddon [:gsnedders]
Reported 2023-11-28 18:23:58 PST
similar to the analyze-layout-tests-results step (c.f., https://github.com/WebKit/WebKit/blob/5965a5a92e802141a8ec48b1a83c9bb88069f1e1/Tools/CISupport/ews-build/steps.py#L4454), we should not *just* take the initial layout-tests run as the definitive set of results this should vastly reduce the number of failed EWS runs as a result of running the tool, and avoid changing tests because they are ungardened on main.
Attachments
Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2023-11-28 18:24:07 PST
<rdar://problem/118909142>
Sam Sneddon [:gsnedders]
Comment 2 2023-12-14 16:55:58 PST
https://github.com/WebKit/WebKit/pull/21838 is laying some of the group work there, and the comment I just posted there explains where I expect to build on top of that PR: > The end goal is to move to having some class representing a group of runs, consisting of one or more layout tests runs plus zero or more runs without changes. We can then have that class tell us "what are the new (stable) failures" and use that.
Sam Sneddon [:gsnedders]
Comment 3 2023-12-16 02:57:55 PST
this also probably needs to either deal directly with the results database (which requires knowing the base revision), or pull the data from the EWS run, as we often ignore (and even avoid re-runs) test failures based on it: https://ews-build.webkit.org/#/builders/36/builds/27897
Note You need to log in before you can comment on or make changes to this bug.