Bug 203494 – results.webkit.org: Show results which are missing expectations

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Jonathan Bedard]

More generally, we should have a visual difference between 'WARNING' and 'PASS'.

[Jonathan Bedard]

<rdar://problem/56436643>

[Jonathan Bedard]

Created attachment 382085 [details]
Patch

[Aakash Jain]

Comment on attachment 382085 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=382085&action=review

> Tools/ChangeLog:3
> +        results.webkit.org: Show missing results

Bug title seems different from the patch. The patch seems to be about adding support for tests having 'warning' status.

[Jonathan Bedard]

(In reply to Aakash Jain from comment #3)
> Comment on attachment 382085 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=382085&action=review
> 
> > Tools/ChangeLog:3
> > +        results.webkit.org: Show missing results
> 
> Bug title seems different from the patch. The patch seems to be about adding
> support for tests having 'warning' status.

'WARNING' is the more general case. Missing results are a special type of warning, the last database assumed that all WARNINGs indicated a missing result, since I've tried to design the results database to be applicable to more than just layout tests, I don't think we want to explicitly say 'Missing result'

[Jonathan Bedard]

Created attachment 382090 [details]
Patch

[Jonathan Bedard]

Created attachment 382093 [details]
Screen shot

[Zhifei Fang]

I think using '!' for warning is make more sense, however we already used it for crash tests, maybe I should draw two new icons make this more clear.

[Jonathan Bedard]

(In reply to Zhifei Fang from comment #7)
> I think using '!' for warning is make more sense, however we already used it
> for crash tests, maybe I should draw two new icons make this more clear.

I'm open to new symbols, but given that at the moment, WARNING means a missing test expectation, I think ? makes the most sense.

[Alexey Proskuryakov]

It would certainly be ideal to tell exactly what happened in layout test case. It will be challenging for a new user to figure out what we are implying here. Also, missing results is more like a hard error than a failure.

Landing with the proposed behavior would be an improvement anyway.

[WebKit Commit Bot]

Comment on attachment 382090 [details]
Patch

Clearing flags on attachment: 382090

Committed r251670: <https://trac.webkit.org/changeset/251670>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.