Bug 81812

Summary: garden-o-matic should special case reftests
Product: WebKit Reporter: Ojan Vafai <ojan>
Component: Tools / TestsAssignee: Ojan Vafai <ojan>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, dglazkov
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 69227    
Attachments:
Description Flags
Patch
none
Patch abarth: review+

Description Ojan Vafai 2012-03-21 11:45:40 PDT
Reftests should:
-not have a rebaseline button
-Show somehow that they are reftests
-Show the output of the test, the reference and the diff
Comment 1 Adam Barth 2012-03-21 13:40:26 PDT
I don't plan to implement this feature any time soon, but other folks should feel free to.
Comment 2 Ojan Vafai 2012-03-26 13:54:53 PDT
Created attachment 133886 [details]
Patch
Comment 3 Ojan Vafai 2012-03-26 13:56:23 PDT
Comment on attachment 133886 [details]
Patch

Whoops. Forgot to add test.
Comment 4 Ojan Vafai 2012-03-26 14:14:03 PDT
Created attachment 133890 [details]
Patch
Comment 5 Adam Barth 2012-03-26 14:33:00 PDT
Comment on attachment 133890 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=133890&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/scripts/ui/results.js:177
> +    for (var builder in results) {

We've been using forEach rather than loops.

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/scripts/ui/results.js:203
> +                $(header).append('<div class="non-action-button">Reftest cannot be rebaselined</div>');

What is the gardener supposed to do in this case?  Can we provide some help text?
Comment 6 Ojan Vafai 2012-03-26 14:40:53 PDT
(In reply to comment #5)
> (From update of attachment 133890 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=133890&action=review
> 
> > Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/scripts/ui/results.js:203
> > +                $(header).append('<div class="non-action-button">Reftest cannot be rebaselined</div>');
> 
> What is the gardener supposed to do in this case?  Can we provide some help text?

I'm not really sure. Don't have enough experience with reftests to understand what the common pitfalls/solutions are. Open to suggestions.
Comment 7 Adam Barth 2012-03-26 14:42:27 PDT
> I'm not really sure. Don't have enough experience with reftests to understand what the common pitfalls/solutions are. Open to suggestions.

Maybe we should say that?  We could encourage folks to who run into these situations to email the gardening lists with thoughts about what we should do in these cases.
Comment 8 Ojan Vafai 2012-03-26 14:56:08 PDT
Committed r112149: <http://trac.webkit.org/changeset/112149>