Bug 29263 – Add option to run-webkit-tests to repeat the tests N times

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Simon Fraser (smfr)]

It would be useful in tracking down random crashers to be able to have run-webkit-tests repeat the tests N times.

[Simon Fraser (smfr)]

Created attachment 39580 [details]
Patch

[Darin Adler]

Comment on attachment 39580 [details]
Patch

> +if ($iterations) {
> +    my @orig_tests = @tests;
> +    for (my $i = 1; $i < $iterations; $i++) {
> +        push(@tests, @orig_tests);
> +    }
> +}

No need for the if -- it works fine without it and just copies the array once.

We normally name these things like @originalTests rather than $origTests.

r=me

[Darin Adler]

Comment on attachment 39580 [details]
Patch

@originalTests rather than @orig_tests I meant

[Alexey Proskuryakov]

*** Bug 29220 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

http://trac.webkit.org/changeset/48378

[Eric Seidel (no email)]

I'm confused by the flag description as I'm not certain what it will do when not passed a single test.  Will it run each in order and then repeat the same ordered set, or will it run each one N times before running the next?

[Simon Fraser (smfr)]

It will run them like:
ABCABCABC
rather than AAABBBCCC

[Eric Seidel (no email)]

(In reply to comment #7)
> It will run them like:
> ABCABCABC
> rather than AAABBBCCC

Ok.  I think that mode is useful, but I think it's not very useful for finding the root cause of bugs like bug 28624.  In bug 28624 one test is causing later tests to crash.  ABCABCABC won't reveal which test is causing the later ones to crash, bug AAABBBCCC will reveal some sets of similar bugs (but not all).

Either way, I think this is awesome support to have!  Both types of repeated test runs are useful.  I think we should probably undup bug 29220 and have that cover adding AAABBBCCC support.

[Eric Seidel (no email)]

(In reply to comment #8)
> Ok.  I think that mode is useful, but I think it's not very useful for finding
> the root cause of bugs like bug 28624.  In bug 28624 one test is causing later
> tests to crash.  ABCABCABC won't reveal which test is causing the later ones to
> crash, bug AAABBBCCC will reveal some sets of similar bugs (but not all).

That was way more negative sounding than I meant.  This support is awesome.  Period.

Thank you!