Bug 254149 - [Results Dashboard] List failing results
Summary: [Results Dashboard] List failing results
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-03-20 03:36 PDT by Sam Sneddon [:gsnedders]
Modified: 2023-03-20 08:22 PDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sam Sneddon [:gsnedders] 2023-03-20 03:36:45 PDT
It seems like at an API level this was implemented in bug 203905, but we don't have any easy way through the UI to just get a list of failing results for a given test.

This is often useful when investigating flakiness, as it's often most useful to just look at the results when the test does fail. I don't want to have to scroll through all the test results to find failing results—I'd just like a list of most recent failures.
Comment 1 Radar WebKit Bug Importer 2023-03-20 03:36:55 PDT
<rdar://problem/106937015>
Comment 2 Jonathan Bedard 2023-03-20 08:20:14 PDT
I don't think https://bugs.webkit.org/show_bug.cgi?id=203905 is quite the API version of what you're requesting here. It sounds like the request here is basically "give me a list of test runs where a given test has failed", https://bugs.webkit.org/show_bug.cgi?id=203905 is "give me a list of all tests which have failed in a given range".

That being said, I do think the API gives us everything we need to build a UI for "give me a list of test runs where a given test has failed" easily, I'm just not quite sure what the user story to get to such a view would be. I don't think such a UI would make much sense as a timeline (it would involve much scrolling), I'm thinking either a collapsed timeline which eliminates commits that weren't testing (something Zhifei and I have discussed in the past) or something akin to our "investigate" tab which gives you a list of builds.
Comment 3 Sam Sneddon [:gsnedders] 2023-03-20 08:22:10 PDT
(In reply to Jonathan Bedard from comment #2)
> I don't think https://bugs.webkit.org/show_bug.cgi?id=203905 is quite the
> API version of what you're requesting here. It sounds like the request here
> is basically "give me a list of test runs where a given test has failed",
> https://bugs.webkit.org/show_bug.cgi?id=203905 is "give me a list of all
> tests which have failed in a given range".

Ah, I misunderstood from skimming what that issue was!

> That being said, I do think the API gives us everything we need to build a
> UI for "give me a list of test runs where a given test has failed" easily,
> I'm just not quite sure what the user story to get to such a view would be.
> I don't think such a UI would make much sense as a timeline (it would
> involve much scrolling), I'm thinking either a collapsed timeline which
> eliminates commits that weren't testing (something Zhifei and I have
> discussed in the past) or something akin to our "investigate" tab which
> gives you a list of builds.

Yeah, I'm not sure what the right UI (and path to the UI) is either.