Bug 203494 - results.webkit.org: Show results which are missing expectations
Summary: results.webkit.org: Show results which are missing expectations
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-10-28 11:15 PDT by Jonathan Bedard
Modified: 2019-10-28 13:50 PDT (History)
5 users (show)

See Also:

Attachments
Patch (3.71 KB, patch)
2019-10-28 11:19 PDT, Jonathan Bedard 		no flags Details | Formatted Diff | Diff
Patch (3.74 KB, patch)
2019-10-28 11:45 PDT, Jonathan Bedard 		no flags Details | Formatted Diff | Diff
Screen shot (304.90 KB, image/png)
2019-10-28 11:47 PDT, Jonathan Bedard 		no flags Details
Show Obsolete (1) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2019-10-28 11:15:19 PDT 
More generally, we should have a visual difference between 'WARNING' and 'PASS'.
Comment 1 Jonathan Bedard 2019-10-28 11:15:42 PDT 
<rdar://problem/56436643>
Comment 2 Jonathan Bedard 2019-10-28 11:19:47 PDT 
Created attachment 382085 [details]
Patch
Comment 3 Aakash Jain 2019-10-28 11:25:33 PDT 
Comment on attachment 382085 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=382085&action=review

> Tools/ChangeLog:3
> +        results.webkit.org: Show missing results

Bug title seems different from the patch. The patch seems to be about adding support for tests having 'warning' status.
Comment 4 Jonathan Bedard 2019-10-28 11:35:02 PDT 
(In reply to Aakash Jain from comment #3)
> Comment on attachment 382085 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=382085&action=review
> 
> > Tools/ChangeLog:3
> > +        results.webkit.org: Show missing results
> 
> Bug title seems different from the patch. The patch seems to be about adding
> support for tests having 'warning' status.

'WARNING' is the more general case. Missing results are a special type of warning, the last database assumed that all WARNINGs indicated a missing result, since I've tried to design the results database to be applicable to more than just layout tests, I don't think we want to explicitly say 'Missing result'
Comment 5 Jonathan Bedard 2019-10-28 11:45:42 PDT 
Created attachment 382090 [details]
Patch
Comment 6 Jonathan Bedard 2019-10-28 11:47:41 PDT 
Created attachment 382093 [details]
Screen shot
Comment 7 Zhifei Fang 2019-10-28 11:54:05 PDT 
I think using '!' for warning is make more sense, however we already used it for crash tests, maybe I should draw two new icons make this more clear.
Comment 8 Jonathan Bedard 2019-10-28 12:53:38 PDT 
(In reply to Zhifei Fang from comment #7)
> I think using '!' for warning is make more sense, however we already used it
> for crash tests, maybe I should draw two new icons make this more clear.

I'm open to new symbols, but given that at the moment, WARNING means a missing test expectation, I think ? makes the most sense.
Comment 9 Alexey Proskuryakov 2019-10-28 13:00:24 PDT 
It would certainly be ideal to tell exactly what happened in layout test case. It will be challenging for a new user to figure out what we are implying here. Also, missing results is more like a hard error than a failure.

Landing with the proposed behavior would be an improvement anyway.
Comment 10 WebKit Commit Bot 2019-10-28 13:50:54 PDT 
Comment on attachment 382090 [details]
Patch

Clearing flags on attachment: 382090

Committed r251670: <https://trac.webkit.org/changeset/251670>
Comment 11 WebKit Commit Bot 2019-10-28 13:50:56 PDT 
All reviewed patches have been landed.  Closing bug.