Bug 20697 - REGRESSION(r36130-r36247): nsieve sunspider test slower
Summary: REGRESSION(r36130-r36247): nsieve sunspider test slower
Status: RESOLVED WONTFIX
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 528+ (Nightly build)
Hardware: Mac OS X 10.5
: P1 Normal
Assignee: Nobody
URL: http://www2.webkit.org/perf/sunspider...
Keywords:
Depends on:
Blocks:
 
Reported: 2008-09-07 04:20 PDT by Gavin Sherlock
Modified: 2012-03-06 23:58 PST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gavin Sherlock 2008-09-07 04:20:25 PDT
Comparing r36130 and r36247 on Sunspider shows that the nsieve test has regressed by 1.15x (though overall, I see a 1.47x gain, presumably due to the squirrelfish-extreme branch merge, which is fabulous!).  Complete Sunspider results pasted below:

TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           1.47x as fast     1498.0ms +/- 0.3%   1022.2ms +/- 0.5%     significant

=============================================================================

  3d:                  1.36x as fast      216.4ms +/- 3.5%    159.2ms +/- 5.8%     significant
    cube:              1.51x as fast       75.8ms +/- 11.2%     50.2ms +/- 15.0%     significant
    morph:             1.36x as fast       76.2ms +/- 1.8%     56.2ms +/- 1.0%     significant
    raytrace:          1.22x as fast       64.4ms +/- 1.1%     52.8ms +/- 5.4%     significant

  access:              1.57x as fast      197.6ms +/- 1.7%    125.8ms +/- 2.9%     significant
    binary-trees:      1.39x as fast       40.0ms +/- 3.8%     28.8ms +/- 4.7%     significant
    fannkuch:          2.49x as fast       44.8ms +/- 1.2%     18.0ms +/- 0.0%     significant
    nbody:             1.64x as fast       94.2ms +/- 1.7%     57.6ms +/- 1.9%     significant
    nsieve:            *1.15x as slow*     18.6ms +/- 3.7%     21.4ms +/- 13.4%     significant

  bitops:              1.95x as fast      117.8ms +/- 1.9%     60.4ms +/- 2.3%     significant
    3bit-bits-in-byte: 2.27x as fast       20.4ms +/- 3.3%      9.0ms +/- 0.0%     significant
    bits-in-byte:      1.96x as fast       22.4ms +/- 3.0%     11.4ms +/- 6.0%     significant
    bitwise-and:       1.44x as fast       34.0ms +/- 2.6%     23.6ms +/- 2.9%     significant
    nsieve-bits:       2.50x as fast       41.0ms +/- 2.1%     16.4ms +/- 4.2%     significant

  controlflow:         1.13x as fast       15.8ms +/- 3.5%     14.0ms +/- 0.0%     significant
    recursive:         1.13x as fast       15.8ms +/- 3.5%     14.0ms +/- 0.0%     significant

  crypto:              1.51x as fast       82.8ms +/- 0.7%     55.0ms +/- 1.6%     significant
    aes:               1.82x as fast       25.8ms +/- 2.2%     14.2ms +/- 3.9%     significant
    md5:               1.39x as fast       28.6ms +/- 2.4%     20.6ms +/- 3.3%     significant
    sha1:              1.41x as fast       28.4ms +/- 2.4%     20.2ms +/- 2.8%     significant

  date:                1.08x as fast      148.6ms +/- 1.5%    138.2ms +/- 0.8%     significant
    format-tofte:      1.02x as fast       94.0ms +/- 1.3%     92.6ms +/- 0.7%     significant
    format-xparb:      1.20x as fast       54.6ms +/- 3.5%     45.6ms +/- 1.5%     significant

  math:                1.42x as fast      179.0ms +/- 2.6%    126.2ms +/- 1.3%     significant
    cordic:            1.52x as fast       55.2ms +/- 3.7%     36.4ms +/- 1.9%     significant
    partial-sums:      1.38x as fast       90.4ms +/- 2.1%     65.6ms +/- 1.7%     significant
    spectral-norm:     1.38x as fast       33.4ms +/- 3.3%     24.2ms +/- 2.3%     significant

  regexp:              4.92x as fast      185.8ms +/- 0.7%     37.8ms +/- 1.5%     significant
    dna:               4.92x as fast      185.8ms +/- 0.7%     37.8ms +/- 1.5%     significant

  string:              1.16x as fast      354.2ms +/- 1.0%    305.6ms +/- 1.5%     significant
    base64:            1.27x as fast       29.0ms +/- 0.0%     22.8ms +/- 2.4%     significant
    fasta:             1.07x as fast       63.8ms +/- 1.6%     59.6ms +/- 1.1%     significant
    tagcloud:          1.12x as fast      102.8ms +/- 0.5%     91.8ms +/- 1.1%     significant
    unpack-code:       1.20x as fast       93.6ms +/- 2.0%     78.2ms +/- 2.8%     significant
    validate-input:    1.22x as fast       65.0ms +/- 1.9%     53.2ms +/- 3.5%     significant
Comment 1 Gavin Sherlock 2008-09-07 04:21:23 PDT
Tested on  a MacPro with 2 x 2.66GHz dual core intel xeons, with 2GB RAM on 10.5.4
Comment 2 Maciej Stachowiak 2008-09-07 15:11:10 PDT
I did not see a regression on nsieve comparing before to after, but obviously we should double check.
Comment 3 Gavin Barraclough 2012-03-06 23:58:15 PST
Some regressions show on some machines but not others.  I expect this tracked something useful at the time, but enough has changed 74000 revisions (!) that I think it's safe to say we're no longer tracking this as a regression!  Thanks for the bug report.

Not sure what the right bug resolution is for far-too-much-water-has-passed-under-the-bridge-to-do-anything-about-this, I'm guessing WONTFIX is the closest.