29263 2009-09-14 17:39:45 -0700 Add option to run-webkit-tests to repeat the tests N times 2009-09-15 11:01:29 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 RESOLVED FIXED P2 Normal --- 1 simon.fraser simon.fraser eric oldest_to_newest 147229 0 simon.fraser 2009-09-14 17:39:45 -0700 It would be useful in tracking down random crashers to be able to have run-webkit-tests repeat the tests N times. 147230 1 39580 simon.fraser 2009-09-14 17:42:38 -0700 Created attachment 39580 Patch 147233 2 39580 darin 2009-09-14 17:48:07 -0700 Comment on attachment 39580 Patch > +if ($iterations) { > + my @orig_tests = @tests; > + for (my $i = 1; $i < $iterations; $i++) { > + push(@tests, @orig_tests); > + } > +} No need for the if -- it works fine without it and just copies the array once. We normally name these things like @originalTests rather than $origTests. r=me 147234 3 39580 darin 2009-09-14 17:48:24 -0700 Comment on attachment 39580 Patch @originalTests rather than @orig_tests I meant 147325 4 ap 2009-09-15 10:46:39 -0700 *** Bug 29220 has been marked as a duplicate of this bug. *** 147327 5 simon.fraser 2009-09-15 10:50:11 -0700 http://trac.webkit.org/changeset/48378 147329 6 eric 2009-09-15 10:51:44 -0700 I'm confused by the flag description as I'm not certain what it will do when not passed a single test. Will it run each in order and then repeat the same ordered set, or will it run each one N times before running the next? 147333 7 simon.fraser 2009-09-15 10:54:59 -0700 It will run them like: ABCABCABC rather than AAABBBCCC 147337 8 eric 2009-09-15 11:00:48 -0700 (In reply to comment #7) > It will run them like: > ABCABCABC > rather than AAABBBCCC Ok. I think that mode is useful, but I think it's not very useful for finding the root cause of bugs like bug 28624. In bug 28624 one test is causing later tests to crash. ABCABCABC won't reveal which test is causing the later ones to crash, bug AAABBBCCC will reveal some sets of similar bugs (but not all). Either way, I think this is awesome support to have! Both types of repeated test runs are useful. I think we should probably undup bug 29220 and have that cover adding AAABBBCCC support. 147338 9 eric 2009-09-15 11:01:29 -0700 (In reply to comment #8) > Ok. I think that mode is useful, but I think it's not very useful for finding > the root cause of bugs like bug 28624. In bug 28624 one test is causing later > tests to crash. ABCABCABC won't reveal which test is causing the later ones to > crash, bug AAABBBCCC will reveal some sets of similar bugs (but not all). That was way more negative sounding than I meant. This support is awesome. Period. Thank you! 39580 2009-09-14 17:42:38 -0700 2009-09-14 17:48:24 -0700 Patch 29263_patch.txt text/plain 2286 simon.fraser ZGlmZiAtLWdpdCBhL1dlYktpdFRvb2xzL0NoYW5nZUxvZyBiL1dlYktpdFRvb2xzL0NoYW5nZUxv ZwppbmRleCA5YmQ2NGY2Li41ZTI3NDg5IDEwMDY0NAotLS0gYS9XZWJLaXRUb29scy9DaGFuZ2VM b2cKKysrIGIvV2ViS2l0VG9vbHMvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMTQgQEAKKzIwMDktMDkt MTQgIFNpbW9uIEZyYXNlciAgPHNpbW9uLmZyYXNlckBhcHBsZS5jb20+CisKKyAgICAgICAgUmV2 aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisgICAgICAgIAorICAgICAgICBBZGQgLS1pdGVyYXRp b25zIG9wdGlvbiB0byBydW4td2Via2l0LXRlc3RzIHRvIHJlcGVhdCB0aGUgdGVzdHMgTiB0aW1l cworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjkyNjMK KworICAgICAgICBXaGVuIHJ1biB3aXRoIC0taXRlcmF0aW9ucyBOLCBydW4td2Via2l0LXRlc3Rz IHdpbGwgcmVwZWF0IHRoZSB0ZXN0cyBOIHRpbWVzLgorCisgICAgICAgICogU2NyaXB0cy9ydW4t d2Via2l0LXRlc3RzOgorCiAyMDA5LTA5LTEyICBNYXJrIFJvd2UgIDxtcm93ZUBhcHBsZS5jb20+ CiAKICAgICAgICAgUmV2aWV3ZWQgYnkgRGFuIEJlcm5zdGVpbi4KZGlmZiAtLWdpdCBhL1dlYktp dFRvb2xzL1NjcmlwdHMvcnVuLXdlYmtpdC10ZXN0cyBiL1dlYktpdFRvb2xzL1NjcmlwdHMvcnVu LXdlYmtpdC10ZXN0cwppbmRleCBjYWJlMGQ4Li4xYmJjNzFjIDEwMDc1NQotLS0gYS9XZWJLaXRU b29scy9TY3JpcHRzL3J1bi13ZWJraXQtdGVzdHMKKysrIGIvV2ViS2l0VG9vbHMvU2NyaXB0cy9y dW4td2Via2l0LXRlc3RzCkBAIC0xMTYsNiArMTE2LDcgQEAgbXkgJGd1YXJkTWFsbG9jID0gJyc7 CiBteSAkaHR0cGRQb3J0ID0gODAwMDsKIG15ICRodHRwZFNTTFBvcnQgPSA4NDQzOwogbXkgJGln bm9yZVRlc3RzID0gJyc7CitteSAkaXRlcmF0aW9ucyA9IDE7CiBteSAkbGF1bmNoU2FmYXJpID0g MTsKIG15ICRwbGF0Zm9ybTsKIG15ICRwaXhlbFRlc3RzID0gJyc7CkBAIC0yMDgsNiArMjA5LDcg QEAgVXNhZ2U6ICRwcm9ncmFtTmFtZSBbb3B0aW9uc10gW3Rlc3RkaXJ8dGVzdHBhdGggLi4uXQog ICAtaHwtLWhlbHAgICAgICAgICAgICAgICAgICAgICAgIFNob3cgdGhpcyBoZWxwIG1lc3NhZ2UK ICAgLS1bbm8tXWh0dHAgICAgICAgICAgICAgICAgICAgICBSdW4gKG9yIGRvIG5vdCBydW4pIGh0 dHAgdGVzdHMgKGRlZmF1bHQ6ICRodHRwRGVmYXVsdCkKICAgLWl8LS1pZ25vcmUtdGVzdHMgICAg ICAgICAgICAgICBDb21tYS1zZXBhcmF0ZWQgbGlzdCBvZiBkaXJlY3RvcmllcyBvciB0ZXN0cyB0 byBpZ25vcmUKKyAgLS1pdGVyYXRpb25zIG4gICAgICAgICAgICAgICAgICBOdW1iZXIgb2YgdGlt ZXMgdG8gcnVuIHRoZSB0ZXN0cwogICAtLVtuby1dbGF1bmNoLXNhZmFyaSAgICAgICAgICAgIExh dW5jaCAob3IgZG8gbm90IGxhdW5jaCkgU2FmYXJpIHRvIGRpc3BsYXkgdGVzdCByZXN1bHRzIChk ZWZhdWx0OiAkbGF1bmNoU2FmYXJpRGVmYXVsdCkKICAgLWx8LS1sZWFrcyAgICAgICAgICAgICAg ICAgICAgICBFbmFibGUgbGVha3MgY2hlY2tpbmcKICAgLS1bbm8tXW5ldy10ZXN0LXJlc3VsdHMg ICAgICAgICBHZW5lcmF0ZSByZXN1bHRzIGZvciBuZXcgdGVzdHMKQEAgLTI0OCw2ICsyNTAsNyBA QCBteSAkZ2V0T3B0aW9uc1Jlc3VsdCA9IEdldE9wdGlvbnMoCiAgICAgJ2hlbHB8aCcgPT4gXCRz aG93SGVscCwKICAgICAnaHR0cCEnID0+IFwkdGVzdEhUVFAsCiAgICAgJ2lnbm9yZS10ZXN0c3xp PXMnID0+IFwkaWdub3JlVGVzdHMsCisgICAgJ2l0ZXJhdGlvbnM9aScgPT4gXCRpdGVyYXRpb25z LAogICAgICdsYXVuY2gtc2FmYXJpIScgPT4gXCRsYXVuY2hTYWZhcmksCiAgICAgJ2xlYWtzfGwn ID0+IFwkc2hvdWxkQ2hlY2tMZWFrcywKICAgICAncGl4ZWwtdGVzdHN8cCcgPT4gXCRwaXhlbFRl c3RzLApAQCAtNTgwLDYgKzU4MywxNCBAQCBteSAlZXhwZWN0ZWRSZXN1bHRQYXRoczsKICMgU2h1 ZmZsZSB0aGUgYXJyYXkKIEB0ZXN0cyA9IHNodWZmbGUoQHRlc3RzKSBpZiAkcmFuZG9taXplVGVz dHM7CiAKKyMgQWRkIGl0ZXJhdGlvbnMKK2lmICgkaXRlcmF0aW9ucykgeworICAgIG15IEBvcmln X3Rlc3RzID0gQHRlc3RzOworICAgIGZvciAobXkgJGkgPSAxOyAkaSA8ICRpdGVyYXRpb25zOyAk aSsrKSB7CisgICAgICAgIHB1c2goQHRlc3RzLCBAb3JpZ190ZXN0cyk7CisgICAgfQorfQorCiBm b3IgbXkgJHRlc3QgKEB0ZXN0cykgewogICAgIG5leHQgaWYgJHRlc3QgZXEgJ3Jlc3VsdHMuaHRt bCc7CiAK