Bug 238809 - [resultsdb] Expose reported flakiness information in the UI
Summary: [resultsdb] Expose reported flakiness information in the UI
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Angelos Oikonomopoulos
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-04-05 09:12 PDT by Angelos Oikonomopoulos
Modified: 2022-07-08 10:38 PDT (History)
2 users (show)

See Also:


Attachments
Patch (10.53 KB, patch)
2022-04-05 09:17 PDT, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Screenshot of the suites view with Show number of flaky tests on (87.79 KB, image/png)
2022-04-05 13:55 PDT, Angelos Oikonomopoulos
no flags Details
Screenshot of the search view with Show test flakiness on (76.73 KB, image/png)
2022-04-05 13:57 PDT, Angelos Oikonomopoulos
no flags Details
Patch (10.72 KB, patch)
2022-04-07 05:38 PDT, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Patch (10.70 KB, patch)
2022-04-11 04:35 PDT, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Patch (10.86 KB, patch)
2022-04-14 05:39 PDT, Angelos Oikonomopoulos
jbedard: review+
ews-feeder: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Angelos Oikonomopoulos 2022-04-05 09:12:35 PDT
[resultsdb] Expose reported flakiness information in the UI
Comment 1 Angelos Oikonomopoulos 2022-04-05 09:17:49 PDT
Created attachment 456707 [details]
Patch
Comment 2 Angelos Oikonomopoulos 2022-04-05 09:29:32 PDT
FWIW I tried using the dotWidth as the maxWidth to CanvasRenderingContext2D.fillText() but that didn't seem to help (and I suspect the resulting text would be unreadably small in any case).
Comment 3 Jonathan Bedard 2022-04-05 12:43:50 PDT
Comment on attachment 456707 [details]
Patch

Generally looks good, although I'd like to see it before landing. Do you have screen shots?
Comment 4 Angelos Oikonomopoulos 2022-04-05 13:55:07 PDT
Created attachment 456742 [details]
Screenshot of the suites view with Show number of flaky tests on

This is a screenshot after browsing to /suites (with some randomly generated test results inserted) and turning on 'Show number of flaky tests' (defaults to off).
Comment 5 Angelos Oikonomopoulos 2022-04-05 13:57:16 PDT
Created attachment 456743 [details]
Screenshot of the search view with Show test flakiness on

Visiting /?suite=javascriptcore-tests&test=stress/test1 and setting 'Show test flakiness' to on (defaults to off).
Comment 6 Angelos Oikonomopoulos 2022-04-07 05:38:08 PDT
Created attachment 456914 [details]
Patch
Comment 7 Angelos Oikonomopoulos 2022-04-08 01:31:23 PDT
The new version of the patch no longer expects a nested dict, to match the change in https://bugs.webkit.org/show_bug.cgi?id=238806.

I've been wondering whether it makes sense to have a radio button for the dot tags, so that it's both always apparent what piece of information one is looking at (so there would be no need for additional text -- e.g. the current patch prepends 'fl ' to the number of flaky tests). This would also mean that the tag text can't get too long and start overlapping with adjacent tags.
Comment 8 Jonathan Bedard 2022-04-08 16:21:05 PDT
Comment on attachment 456914 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=456914&action=review

I would like to get https://bugs.webkit.org/show_bug.cgi?id=238806 landed first so we can deploy this on the staging instance before landing

> Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js:733
> +                    tag.push('fl ' + data.details.number_of_flaky_tests);

This ends up being a bit long in the UI...I suppose it's hidden behind a switch, so it's not a huge deal,

> Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js:1171
> +                <label>Show number of flaky tests</label>

The name of this label is a bit long. Maybe "Show number of flakes"? Or just "Number of flakes"?
Comment 9 Angelos Oikonomopoulos 2022-04-11 04:35:39 PDT
Created attachment 457246 [details]
Patch
Comment 10 Angelos Oikonomopoulos 2022-04-11 05:11:53 PDT
(In reply to Jonathan Bedard from comment #8)
> Comment on attachment 456914 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=456914&action=review
> 
> I would like to get https://bugs.webkit.org/show_bug.cgi?id=238806 landed
> first so we can deploy this on the staging instance before landing
> 
> > Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js:733
> > +                    tag.push('fl ' + data.details.number_of_flaky_tests);
> 
> This ends up being a bit long in the UI...I suppose it's hidden behind a
> switch, so it's not a huge deal,

Yah, I don't have a good solution for that, other than perhaps the radio button mentioned above.

> > Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js:1171
> > +                <label>Show number of flaky tests</label>
> 
> The name of this label is a bit long. Maybe "Show number of flakes"? Or just
> "Number of flakes"?

Changed to "Number of flakes".
Comment 11 Radar WebKit Bug Importer 2022-04-12 09:13:16 PDT
<rdar://problem/91630047>
Comment 12 Jonathan Bedard 2022-04-13 14:25:32 PDT
Comment on attachment 457246 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=457246&action=review

> Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js:-727
> -                return drawDot(context, x, y, false, tag ? tag : null, symbol, false, color);

New code will show 0, which is not what we want. Probably need an extra check when building tag list to throw out zeros.
Comment 13 Angelos Oikonomopoulos 2022-04-14 05:39:22 PDT
Created attachment 457615 [details]
Patch
Comment 14 Angelos Oikonomopoulos 2022-04-14 05:40:42 PDT
(In reply to Jonathan Bedard from comment #12)
> Comment on attachment 457246 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=457246&action=review
> 
> > Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js:-727
> > -                return drawDot(context, x, y, false, tag ? tag : null, symbol, false, color);
> 
> New code will show 0, which is not what we want. Probably need an extra
> check when building tag list to throw out zeros.

Oops. Hopefully fixed now.
Comment 15 Angelos Oikonomopoulos 2022-06-30 03:23:14 PDT
Ping.
Comment 16 Jonathan Bedard 2022-07-01 12:39:42 PDT
Finally got a chance to deploy this on our staging instance today, looks good to go! Change will need to be re-uploaded with a commit message instead of a changelog, but the code itself looks good!
Comment 17 Angelos Oikonomopoulos 2022-07-04 02:45:24 PDT
Pull request: https://github.com/WebKit/WebKit/pull/2051
Comment 18 EWS 2022-07-04 05:18:44 PDT
Committed 252114@main (654dea109d94): <https://commits.webkit.org/252114@main>

Reviewed commits have been landed. Closing PR #2051 and removing active labels.
Comment 19 Angelos Oikonomopoulos 2022-07-04 05:19:53 PDT
(In reply to Jonathan Bedard from comment #16)
> Finally got a chance to deploy this on our staging instance today, looks
> good to go! Change will need to be re-uploaded with a commit message instead
> of a changelog, but the code itself looks good!

Re-uploaded, thanks for testing it!
Comment 20 Jonathan Bedard 2022-07-05 08:05:44 PDT
(In reply to Angelos Oikonomopoulos from comment #19)
> (In reply to Jonathan Bedard from comment #16)
> > Finally got a chance to deploy this on our staging instance today, looks
> > good to go! Change will need to be re-uploaded with a commit message instead
> > of a changelog, but the code itself looks good!
> 
> Re-uploaded, thanks for testing it!

The problem is the ChangeLog. WebKit no longer has changelogs, only commit messages.
Comment 21 Jonathan Bedard 2022-07-07 07:57:56 PDT
Landed in 252114@main
Comment 22 Jonathan Bedard 2022-07-08 10:38:10 PDT
This has been deployed!