Bug 81812 - garden-o-matic should special case reftests
Summary: garden-o-matic should special case reftests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ojan Vafai
URL:
Keywords:
Depends on:
Blocks: 69227
  Show dependency treegraph
 
Reported: 2012-03-21 11:45 PDT by Ojan Vafai
Modified: 2012-03-26 14:56 PDT (History)
2 users (show)

See Also:


Attachments
Patch (4.86 KB, patch)
2012-03-26 13:54 PDT, Ojan Vafai
no flags Details | Formatted Diff | Diff
Patch (6.71 KB, patch)
2012-03-26 14:14 PDT, Ojan Vafai
abarth: review+
ojan: commit-queue?
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ojan Vafai 2012-03-21 11:45:40 PDT
Reftests should:
-not have a rebaseline button
-Show somehow that they are reftests
-Show the output of the test, the reference and the diff
Comment 1 Adam Barth 2012-03-21 13:40:26 PDT
I don't plan to implement this feature any time soon, but other folks should feel free to.
Comment 2 Ojan Vafai 2012-03-26 13:54:53 PDT
Created attachment 133886 [details]
Patch
Comment 3 Ojan Vafai 2012-03-26 13:56:23 PDT
Comment on attachment 133886 [details]
Patch

Whoops. Forgot to add test.
Comment 4 Ojan Vafai 2012-03-26 14:14:03 PDT
Created attachment 133890 [details]
Patch
Comment 5 Adam Barth 2012-03-26 14:33:00 PDT
Comment on attachment 133890 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=133890&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/scripts/ui/results.js:177
> +    for (var builder in results) {

We've been using forEach rather than loops.

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/scripts/ui/results.js:203
> +                $(header).append('<div class="non-action-button">Reftest cannot be rebaselined</div>');

What is the gardener supposed to do in this case?  Can we provide some help text?
Comment 6 Ojan Vafai 2012-03-26 14:40:53 PDT
(In reply to comment #5)
> (From update of attachment 133890 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=133890&action=review
> 
> > Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/scripts/ui/results.js:203
> > +                $(header).append('<div class="non-action-button">Reftest cannot be rebaselined</div>');
> 
> What is the gardener supposed to do in this case?  Can we provide some help text?

I'm not really sure. Don't have enough experience with reftests to understand what the common pitfalls/solutions are. Open to suggestions.
Comment 7 Adam Barth 2012-03-26 14:42:27 PDT
> I'm not really sure. Don't have enough experience with reftests to understand what the common pitfalls/solutions are. Open to suggestions.

Maybe we should say that?  We could encourage folks to who run into these situations to email the gardening lists with thoughts about what we should do in these cases.
Comment 8 Ojan Vafai 2012-03-26 14:56:08 PDT
Committed r112149: <http://trac.webkit.org/changeset/112149>