Bug 29220 - run-webkit-tests needs a --repeat-each=N option (AAABBBCCC instead of ABCABCABC)
Summary: run-webkit-tests needs a --repeat-each=N option (AAABBBCCC instead of ABCABCABC)
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-09-11 19:02 PDT by Eric Seidel (no email)
Modified: 2009-09-22 15:39 PDT (History)
1 user (show)

See Also:


Attachments
Patch v1 (2.81 KB, patch)
2009-09-22 13:56 PDT, Eric Seidel (no email)
darin: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2009-09-11 19:02:25 PDT
run-webkit-tests needs a --repeat-each=N option which will repeat each test N times.

so:

run-webkit-tests foo.html bar.html --repeat-each=N
would do:

foo -> passed
foo -> passed
bar -> passed
bar -> passed

This is useful for finding flakey tests (like https://bugs.webkit.org/show_bug.cgi?id=28845#c35) or tests which are causing later tests to fail.  It also could be useful for finding tests which fail if their resources are already cached, etc.
Comment 1 Alexey Proskuryakov 2009-09-15 10:46:39 PDT

*** This bug has been marked as a duplicate of bug 29263 ***
Comment 2 Simon Fraser (smfr) 2009-09-15 10:51:15 PDT
Actually in 29263 I took the easy way out and just duplicated the @tests array N times, so that the order will be:

ABCABCABC

not

AAABBBCCC

I'd be fine either way.
Comment 3 Eric Seidel (no email) 2009-09-15 11:03:17 PDT
After discussion in bug 29263, unduping...

bug 29263 added support for running tests ABC as ABCABCABC
this bug is about running test ABC as AAABBBCCC
Comment 4 Eric Seidel (no email) 2009-09-22 13:56:52 PDT
Created attachment 39941 [details]
Patch v1
Comment 5 Darin Adler 2009-09-22 14:07:21 PDT
Comment on attachment 39941 [details]
Patch v1

> -  --iterations n                  Number of times to run the tests
> +  --iterations n                  Number of times to run the set of tests (e.g. ABCABCABC)
> +  --repeat-each n                 Number of times to run each test (e.g. AAABBBCCC)

These are alphabetical, so --repeat-each should go further down in the list.
Comment 6 Eric Seidel (no email) 2009-09-22 14:11:40 PDT
Committed r48649: <http://trac.webkit.org/changeset/48649>
Comment 7 Eric Seidel (no email) 2009-09-22 14:11:44 PDT
Comment on attachment 39941 [details]
Patch v1

Thanks!  Fixed.
Comment 8 Eric Seidel (no email) 2009-09-22 15:39:49 PDT
Committed r48655: <http://trac.webkit.org/changeset/48655>