Bug 203494

Summary: results.webkit.org: Show results which are missing expectations
Product: WebKit Reporter: Jonathan Bedard <jbedard>
Component: Tools / TestsAssignee: Jonathan Bedard <jbedard>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, ap, commit-queue, webkit-bug-importer, zhifei_fang
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch
none
Screen shot none

Jonathan Bedard
Reported 2019-10-28 11:15:19 PDT
More generally, we should have a visual difference between 'WARNING' and 'PASS'.
Attachments
Patch (3.71 KB, patch)
2019-10-28 11:19 PDT, Jonathan Bedard 		no flags
DetailsFormatted DiffDiff
Patch (3.74 KB, patch)
2019-10-28 11:45 PDT, Jonathan Bedard 		no flags
DetailsFormatted DiffDiff
Screen shot (304.90 KB, image/png)
2019-10-28 11:47 PDT, Jonathan Bedard 		no flags
Details
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
Jonathan Bedard
Comment 1 2019-10-28 11:15:42 PDT
<rdar://problem/56436643>
Jonathan Bedard
Comment 2 2019-10-28 11:19:47 PDT
Created attachment 382085 [details] Patch
Aakash Jain
Comment 3 2019-10-28 11:25:33 PDT
Comment on attachment 382085 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=382085&action=review > Tools/ChangeLog:3 > + results.webkit.org: Show missing results Bug title seems different from the patch. The patch seems to be about adding support for tests having 'warning' status.
Jonathan Bedard
Comment 4 2019-10-28 11:35:02 PDT
(In reply to Aakash Jain from comment #3) > Comment on attachment 382085 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=382085&action=review > > > Tools/ChangeLog:3 > > + results.webkit.org: Show missing results > > Bug title seems different from the patch. The patch seems to be about adding > support for tests having 'warning' status. 'WARNING' is the more general case. Missing results are a special type of warning, the last database assumed that all WARNINGs indicated a missing result, since I've tried to design the results database to be applicable to more than just layout tests, I don't think we want to explicitly say 'Missing result'
Jonathan Bedard
Comment 5 2019-10-28 11:45:42 PDT
Created attachment 382090 [details] Patch
Jonathan Bedard
Comment 6 2019-10-28 11:47:41 PDT
Created attachment 382093 [details] Screen shot
Zhifei Fang
Comment 7 2019-10-28 11:54:05 PDT
I think using '!' for warning is make more sense, however we already used it for crash tests, maybe I should draw two new icons make this more clear.
Jonathan Bedard
Comment 8 2019-10-28 12:53:38 PDT
(In reply to Zhifei Fang from comment #7) > I think using '!' for warning is make more sense, however we already used it > for crash tests, maybe I should draw two new icons make this more clear. I'm open to new symbols, but given that at the moment, WARNING means a missing test expectation, I think ? makes the most sense.
Alexey Proskuryakov
Comment 9 2019-10-28 13:00:24 PDT
It would certainly be ideal to tell exactly what happened in layout test case. It will be challenging for a new user to figure out what we are implying here. Also, missing results is more like a hard error than a failure. Landing with the proposed behavior would be an improvement anyway.
WebKit Commit Bot
Comment 10 2019-10-28 13:50:54 PDT
Comment on attachment 382090 [details] Patch Clearing flags on attachment: 382090 Committed r251670: <https://trac.webkit.org/changeset/251670>
WebKit Commit Bot
Comment 11 2019-10-28 13:50:56 PDT
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.