NEW 16849
SunSpider should be rebased to run at least twice as long 
https://bugs.webkit.org/show_bug.cgi?id=16849
Summary SunSpider should be rebased to run at least twice as long
Eric Seidel (no email)
Reported 2008-01-11 18:48:47 PST
SunSpider should be rebased to run at least twice as long SunSpider now runs in under 4 seconds on my (laptop) hardware. The test should be rebased to take at least twice that long to help reduce error rates. Yes, that means it would be *even slower* in other browsers, but my long term goal is to move all browsers to at least 10x their current JavaScript speed. Rebasing the test to take 2x as long is a step in that direction.
Attachments
Add attachment proposed patch, testcase, etc.
Eric Seidel (no email)
Comment 1 2008-01-11 18:49:50 PST
Error rates is not the sole motivator. But certainly with more signal, small amounts of noise in the testing environment matter less.
Darin Fisher (:fishd, Google)
Comment 2 2008-01-17 22:43:19 PST
Perhaps the tests should just be run multiple times, and then compute mean and variance?
Eric Seidel (no email)
Comment 3 2008-01-17 23:49:34 PST
You can already do that with --runs=15 But that doesn't really address the precision question in my mind.
Note You need to log in before you can comment on or make changes to this bug.