Summary: | Incorporate V8 benchmarks in testing | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Gavin Sherlock <gsherloc> | ||||||
Component: | JavaScriptCore | Assignee: | Cameron Zwarich (cpst) <zwarich> | ||||||
Status: | RESOLVED FIXED | ||||||||
Severity: | Normal | CC: | zwarich | ||||||
Priority: | P2 | ||||||||
Version: | 528+ (Nightly build) | ||||||||
Hardware: | All | ||||||||
OS: | OS X 10.5 | ||||||||
URL: | http://code.google.com/apis/v8/run.html | ||||||||
Attachments: |
|
Description
Gavin Sherlock
2008-09-03 00:55:23 PDT
I was thinking of making a SunSpider test suite from the V8 benchmarks so we can use the SunSpider driver, which gives more meaningful information. Created attachment 23159 [details]
Proposed patch
Here are results on my machine with r36082: ============================================ RESULTS (means and 95% confidence intervals) -------------------------------------------- Total: 578.3ms +/- 0.6% -------------------------------------------- v8: 578.3ms +/- 0.6% crypto: 172.3ms +/- 0.5% deltablue: 22.6ms +/- 1.6% earley-boyer: 74.7ms +/- 0.5% raytrace: 300.0ms +/- 0.8% richards: 8.7ms +/- 4.0% This is too wide of a spread, so Maciej and I agreed we should increase the number of iterations in some uniform way. His suggestion was to make them all take some fixed amount of time in V8, since the benchmarks were chosen to highlight V8's strengths. I'll do that later tonight or tomorrow, but I'll clear the review request for now. Created attachment 23165 [details]
Revised proposed patch
I calibrated the benchmarks so that they all run in a roughly equal time with V8. Also, I did not make a separate subdirectory for the v8 benchmarks like the ubench benchmarks. Since there are no categories in the V8 benchmarks, the results look slightly stupid if they are in their own directory.
|