Bug 47218 – [chromium] should have a way to rebaseline tests from the layout test dashboard

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Ojan Vafai]

In the spirit of making test maintenance less work, it out to be possible to choose a test/platform to rebaseline on the layout test dashboard, then it creates a bug and uploads a patch, similar to what "sheriff-bot rollout" does. Then a committing can cq+ it. 

A few ideas about how to make this happen:
1. add "webkit-patch rebaseline --chromium" or something that calls out to the chromium rebaseline tool to create the patch. then it creates the bug and uploads the patch to it.
2. give http://test-results.appspot.com/dashboards/flakiness_dashboard.html a UI for selecting platforms to rebaseline and have it spit out the command-line arguments to patch to webkit-patch rebasline
3. create a bot (sheriff-bot?) to take those arguments and run the webkit-patch command from step 1.

Of course, this doesn't need to be chromium specific, but for now the dashboard and the rebaseline tool are chromium specific. Once we fix those to be generic, this will work for all platforms.

[Adam Barth]

Won't this problem be solved once we have test bots on bugs.chromium.org?  Then you can just use webkit-patch rebaseline like everyone else.

[Adam Barth]

Actually, what i'd really like is a button to do this from the results.html page.  :)

[Ojan Vafai]

> Won't this problem be solved once we have test bots on bugs.chromium.org?  Then you can just use webkit-patch rebaseline like everyone else.

That's possibly true. My memory is that logic of webkit-patch rebaseline is much more naive.

> Actually, what i'd really like is a button to do this from the results.html page.  :)

Sure, we can add it to both right?

[Dimitri Glazkov (Google)]

(In reply to comment #3)
> > Won't this problem be solved once we have test bots on bugs.chromium.org?  Then you can just use webkit-patch rebaseline like everyone else.
> 
> That's possibly true. My memory is that logic of webkit-patch rebaseline is much more naive.
> 
> > Actually, what i'd really like is a button to do this from the results.html page.  :)
> 
> Sure, we can add it to both right?

I would really, really like for webkit-patch rebaseline --platform=chromium to do the same thing as rebaseline-chromium-webkit-tests, but using all the modern machinery and not spitting out gobs of logging code.

[Stephen Chenney]

LayoutTest failures for Chromium are being marked WontFix. The Bug is still accessible and referenced from TestExpectations.