Bug 203068 – --new-baseline doesn't take the device type into account

• Status: NEW
• Component: Tools / Tests

[Simon Fraser (smfr)]

Trying to generate a new baseline for iPad:
> run-webkit-tests --ipad-simulator fast/scrolling/ios/overflow-scroll-inherited.html --new-baseline

< Writing new expected result "platform/ios-simulator-13-wk2/fast/scrolling/ios/overflow-scroll-inherited-expected.txt"

Not what I want. It should go into an ipad directory.

[Jonathan Bedard]

The trouble with this request (and the reason we didn't do this to begin with) is that it's not totally obvious what behavior you want when generating new baselines. The existing model would indicate that you should actually generate the most specific baseline you possibly can, meaning if you had an 'iPad Pro (9.7-inch)' booted, you would create a baselines for 'iPad Pro (9.7-inch)'. In some cases, (like iPhone 7) that would be correct. In other (your case here, iPad) that would be wrong.

My thought here is that we probably need to address <https://bugs.webkit.org/show_bug.cgi?id=203067> first and lean on what was passed via device-type.

[Simon Fraser (smfr)]

Maybe --new-baseline needs arguments?

[Simon Fraser (smfr)]

This is pretty annoying when trying to fix iPad tests. You have to reset results, copy the result to the iPad dir, then git checkout the original result. Yuck.