Bug 47218 - [chromium] should have a way to rebaseline tests from the layout test dashboard
Summary: [chromium] should have a way to rebaseline tests from the layout test dashboard
Status: RESOLVED WONTFIX
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-10-05 15:17 PDT by Ojan Vafai
Modified: 2013-04-09 16:10 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ojan Vafai 2010-10-05 15:17:43 PDT
In the spirit of making test maintenance less work, it out to be possible to choose a test/platform to rebaseline on the layout test dashboard, then it creates a bug and uploads a patch, similar to what "sheriff-bot rollout" does. Then a committing can cq+ it. 

A few ideas about how to make this happen:
1. add "webkit-patch rebaseline --chromium" or something that calls out to the chromium rebaseline tool to create the patch. then it creates the bug and uploads the patch to it.
2. give http://test-results.appspot.com/dashboards/flakiness_dashboard.html a UI for selecting platforms to rebaseline and have it spit out the command-line arguments to patch to webkit-patch rebasline
3. create a bot (sheriff-bot?) to take those arguments and run the webkit-patch command from step 1.

Of course, this doesn't need to be chromium specific, but for now the dashboard and the rebaseline tool are chromium specific. Once we fix those to be generic, this will work for all platforms.
Comment 1 Adam Barth 2010-10-05 15:21:35 PDT
Won't this problem be solved once we have test bots on bugs.chromium.org?  Then you can just use webkit-patch rebaseline like everyone else.
Comment 2 Adam Barth 2010-10-05 15:25:41 PDT
Actually, what i'd really like is a button to do this from the results.html page.  :)
Comment 3 Ojan Vafai 2010-10-05 15:27:19 PDT
> Won't this problem be solved once we have test bots on bugs.chromium.org?  Then you can just use webkit-patch rebaseline like everyone else.

That's possibly true. My memory is that logic of webkit-patch rebaseline is much more naive.

> Actually, what i'd really like is a button to do this from the results.html page.  :)

Sure, we can add it to both right?
Comment 4 Dimitri Glazkov (Google) 2010-10-05 15:35:15 PDT
(In reply to comment #3)
> > Won't this problem be solved once we have test bots on bugs.chromium.org?  Then you can just use webkit-patch rebaseline like everyone else.
> 
> That's possibly true. My memory is that logic of webkit-patch rebaseline is much more naive.
> 
> > Actually, what i'd really like is a button to do this from the results.html page.  :)
> 
> Sure, we can add it to both right?

I would really, really like for webkit-patch rebaseline --platform=chromium to do the same thing as rebaseline-chromium-webkit-tests, but using all the modern machinery and not spitting out gobs of logging code.
Comment 5 Stephen Chenney 2013-04-09 16:10:15 PDT
LayoutTest failures for Chromium are being marked WontFix. The Bug is still accessible and referenced from TestExpectations.