new-run-webkit-tests should tell you when it creates new expectations
Created attachment 99196 [details] Patch
Comment on attachment 99196 [details] Patch OK. No way to test this with our current logging infrastructure I assume?
Comment on attachment 99196 [details] Patch Yeah, if I could figure out how to test it, I could figure how to stop it from spamming the console during testing.
Comment on attachment 99196 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=99196&action=review > Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py:194 > + _log.warning('Writing new expected result "%s"' % result_name) This should be log.info(); these aren't warnings.
Committed r90079: <http://trac.webkit.org/changeset/90079>
Oh, by the way, it occurred to me that if DRT happens to fail on a text-only test in such a way that dumpAsText() doesn't get called and we don't realize the test is test-only, then the code will incorrectly think that the image is missing and generate a new (wrong) baseline for it. This probably doesn't happen all that often, but is an argument for --no-new-test-results being on by default. Maybe this should be filed as a separate bug somewhere, but I'm not sure how we can actually fix the DRT flaw so that we would know from NRWT's side definitively whether a test should be text-only or not.