Bug 16849
| Summary: | SunSpider should be rebased to run at least twice as long | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Eric Seidel (no email) <eric> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | darin, mjs, seth.gaurav |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | Mac | ||
| OS: | OS X 10.4 | ||
Eric Seidel (no email)
SunSpider should be rebased to run at least twice as long
SunSpider now runs in under 4 seconds on my (laptop) hardware. The test should be rebased to take at least twice that long to help reduce error rates. Yes, that means it would be *even slower* in other browsers, but my long term goal is to move all browsers to at least 10x their current JavaScript speed. Rebasing the test to take 2x as long is a step in that direction.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Eric Seidel (no email)
Error rates is not the sole motivator. But certainly with more signal, small amounts of noise in the testing environment matter less.
Darin Fisher (:fishd, Google)
Perhaps the tests should just be run multiple times, and then compute mean and variance?
Eric Seidel (no email)
You can already do that with --runs=15
But that doesn't really address the precision question in my mind.