I frequently want to re-run new-run-webkit-tests and just re-test the set of tests that failed. It would be nice if there was a way to print the last tests out easily or read them into new-run-webkit-tests. The data is in the unexpected_results.json file, so we just have to process it.
Created attachment 53646 [details] Patch
Comment on attachment 53646 [details] Patch Looks useful.
Comment on attachment 53646 [details] Patch > + optparse.make_option("--print-last-results", action="store_true", > + default=False, help="print the tests in the last run that " > + "had unexpected results."), Do we really need an option for this? Lets just always do this. If python had a way of doing "hidden" commands the way the webkit-patch does for commands, I'd be OK with adding all these options. But having them all visible to users of the script by default makes the script harder to use. > + optparse.make_option("--retry-last-results", action="store_true", > + default=False, help="re-try the tests in the last run that " > + "had unexpected results."), Nit: last-results isn't clear without reading the help text or the code. How about --retry-unexpected-failures?
(In reply to comment #3) > (From update of attachment 53646 [details]) > > + optparse.make_option("--print-last-results", action="store_true", > > + default=False, help="print the tests in the last run that " > > + "had unexpected results."), > > Do we really need an option for this? Lets just always do this. If python had a > way of doing "hidden" commands the way the webkit-patch does for commands, I'd > be OK with adding all these options. But having them all visible to users of > the script by default makes the script harder to use. What do you mean by "always do this"? The patch prints the list and exits if that flag is passed in. > > + optparse.make_option("--retry-last-results", action="store_true", > > + default=False, help="re-try the tests in the last run that " > > + "had unexpected results."), > > Nit: last-results isn't clear without reading the help text or the code. How > about --retry-unexpected-failures? Of course, we retry unexpected passes as well. --retry-unexpected-results? I could also live with -failures, as that is the main intent. -- Dirk
Created attachment 53723 [details] Patch
Committed r57843: <http://trac.webkit.org/changeset/57843>