Bug 214085 - [ews] Limit layout tests failures to display in the status-bubble tooltip and buildbot summary
Summary: [ews] Limit layout tests failures to display in the status-bubble tooltip and...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-07-08 08:49 PDT by Aakash Jain
Modified: 2020-07-22 07:50 PDT (History)
4 users (show)

See Also:


Attachments
Screenshot - unreadable build page (904.11 KB, image/png)
2020-07-08 08:49 PDT, Aakash Jain
no flags Details
Screenshot - unreadable status bubble (2.74 MB, image/png)
2020-07-08 09:01 PDT, Aakash Jain
no flags Details
Patch (9.58 KB, patch)
2020-07-08 09:13 PDT, Aakash Jain
no flags Details | Formatted Diff | Diff
Patch (9.63 KB, patch)
2020-07-08 12:10 PDT, Aakash Jain
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2020-07-08 08:49:20 PDT
Created attachment 403784 [details]
Screenshot - unreadable build page

Sometimes a patch can break lot of layout tests, especially after r262379 which removed the limit of 30 failures for webkit-wpt-import-bot@igalia.com

In those cases, the status-bubble tries to display all those failures, and becomes unreadable. Similarly Buildbot's build page (and builder page) display all the failing test names in the summary string, and becomes pretty unreadable. For e.g.: https://ews-build.webkit.org/#/builders/24/builds/21100 has 642 test failures and the build page is unreadable. Screenshots attached.

We should limit the number of failures to display in the status-bubble and summary strings. Complete list of failing tests can still be seen in the logs of the test-run, in the test-failures section, and in the results.html
Comment 1 Aakash Jain 2020-07-08 09:01:40 PDT
Created attachment 403785 [details]
Screenshot - unreadable status bubble
Comment 2 Aakash Jain 2020-07-08 09:13:36 PDT
Created attachment 403788 [details]
Patch
Comment 3 Alexey Proskuryakov 2020-07-08 11:30:39 PDT
Comment on attachment 403788 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=403788&action=review

> Tools/BuildSlaveSupport/ews-build/steps.py:1929
> +        new_failures_string = ', '.join(list(new_failures)[:self.NUM_FAILURES_TO_DISPLAY])

I think that it would be better to do sorted(), to avoid potential confusion caused by a random subset of failures.
Comment 4 Aakash Jain 2020-07-08 12:10:38 PDT
Created attachment 403801 [details]
Patch
Comment 5 EWS 2020-07-08 12:34:23 PDT
Committed r264127: <https://trac.webkit.org/changeset/264127>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 403801 [details].
Comment 6 Radar WebKit Bug Importer 2020-07-08 12:35:20 PDT
<rdar://problem/65236219>
Comment 7 Aakash Jain 2020-07-22 07:50:50 PDT
This change is working fine, e.g.: https://ews-build.webkit.org/#/builders/24/builds/22231