Bug 29263 - Add option to run-webkit-tests to repeat the tests N times
Summary: Add option to run-webkit-tests to repeat the tests N times
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Simon Fraser (smfr)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-09-14 17:39 PDT by Simon Fraser (smfr)
Modified: 2009-09-15 11:01 PDT (History)
1 user (show)

See Also:


Attachments
Patch (2.23 KB, patch)
2009-09-14 17:42 PDT, Simon Fraser (smfr)
darin: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Fraser (smfr) 2009-09-14 17:39:45 PDT
It would be useful in tracking down random crashers to be able to have run-webkit-tests repeat the tests N times.
Comment 1 Simon Fraser (smfr) 2009-09-14 17:42:38 PDT
Created attachment 39580 [details]
Patch
Comment 2 Darin Adler 2009-09-14 17:48:07 PDT
Comment on attachment 39580 [details]
Patch

> +if ($iterations) {
> +    my @orig_tests = @tests;
> +    for (my $i = 1; $i < $iterations; $i++) {
> +        push(@tests, @orig_tests);
> +    }
> +}

No need for the if -- it works fine without it and just copies the array once.

We normally name these things like @originalTests rather than $origTests.

r=me
Comment 3 Darin Adler 2009-09-14 17:48:24 PDT
Comment on attachment 39580 [details]
Patch

@originalTests rather than @orig_tests I meant
Comment 4 Alexey Proskuryakov 2009-09-15 10:46:39 PDT
*** Bug 29220 has been marked as a duplicate of this bug. ***
Comment 5 Simon Fraser (smfr) 2009-09-15 10:50:11 PDT
http://trac.webkit.org/changeset/48378
Comment 6 Eric Seidel (no email) 2009-09-15 10:51:44 PDT
I'm confused by the flag description as I'm not certain what it will do when not passed a single test.  Will it run each in order and then repeat the same ordered set, or will it run each one N times before running the next?
Comment 7 Simon Fraser (smfr) 2009-09-15 10:54:59 PDT
It will run them like:
ABCABCABC
rather than AAABBBCCC
Comment 8 Eric Seidel (no email) 2009-09-15 11:00:48 PDT
(In reply to comment #7)
> It will run them like:
> ABCABCABC
> rather than AAABBBCCC

Ok.  I think that mode is useful, but I think it's not very useful for finding the root cause of bugs like bug 28624.  In bug 28624 one test is causing later tests to crash.  ABCABCABC won't reveal which test is causing the later ones to crash, bug AAABBBCCC will reveal some sets of similar bugs (but not all).

Either way, I think this is awesome support to have!  Both types of repeated test runs are useful.  I think we should probably undup bug 29220 and have that cover adding AAABBBCCC support.
Comment 9 Eric Seidel (no email) 2009-09-15 11:01:29 PDT
(In reply to comment #8)
> Ok.  I think that mode is useful, but I think it's not very useful for finding
> the root cause of bugs like bug 28624.  In bug 28624 one test is causing later
> tests to crash.  ABCABCABC won't reveal which test is causing the later ones to
> crash, bug AAABBBCCC will reveal some sets of similar bugs (but not all).

That was way more negative sounding than I meant.  This support is awesome.  Period.

Thank you!