Bug 25668 – lots of sunspider performance regressions from r41844 to r43444 on PPC G4

• Component: JavaScriptCore
• Status: RESOLVED WONTFIX

WebKit r41844:
http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B323,257,268,275,284%5D,%223d-morph%22:%5B366,322,323,397,329%5D,%223d-raytrace%22:%5B223,220,208,223,220%5D,%22access-binary-trees%22:%5B87,65,70,67,65%5D,%22access-fannkuch%22:%5B148,127,123,124,124%5D,%22access-nbody%22:%5B248,299,298,354,282%5D,%22access-nsieve%22:%5B53,51,50,51,50%5D,%22bitops-3bit-bits-in-byte%22:%5B40,39,42,39,39%5D,%22bitops-bits-in-byte%22:%5B59,58,89,59,62%5D,%22bitops-bitwise-and%22:%5B93,82,81,81,81%5D,%22bitops-nsieve-bits%22:%5B116,114,128,129,114%5D,%22controlflow-recursive%22:%5B34,37,34,34,33%5D,%22crypto-aes%22:%5B86,90,94,72,87%5D,%22crypto-md5%22:%5B74,75,80,77,74%5D,%22crypto-sha1%22:%5B73,72,76,72,77%5D,%22date-format-tofte%22:%5B139,122,138,124,134%5D,%22date-format-xparb%22:%5B224,206,282,288,222%5D,%22math-cordic%22:%5B181,192,201,198,182%5D,%22math-partial-sums%22:%5B327,231,249,314,327%5D,%22math-spectral-norm%22:%5B108,119,110,109,110%5D,%22regexp-dna%22:%5B610,679,705,623,624%5D,%22string-base64%22:%5B93,92,91,103,90%5D,%22string-fasta%22:%5B148,160,226,154,147%5D,%22string-tagcloud%22:%5B393,327,315,392,340%5D,%22string-unpack-code%22:%5B255,252,242,253,339%5D,%22string-validate-input%22:%5B214,228,303,217,218%5D%7D

WebKit r43444:
http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B415,347,343,344,324%5D,%223d-morph%22:%5B352,366,380,376,337%5D,%223d-raytrace%22:%5B255,242,258,268,289%5D,%22access-binary-trees%22:%5B39,37,40,38,39%5D,%22access-fannkuch%22:%5B125,128,127,126,140%5D,%22access-nbody%22:%5B354,401,392,477,416%5D,%22access-nsieve%22:%5B51,50,50,53,50%5D,%22bitops-3bit-bits-in-byte%22:%5B39,39,39,39,40%5D,%22bitops-bits-in-byte%22:%5B69,58,59,58,119%5D,%22bitops-bitwise-and%22:%5B71,67,83,68,66%5D,%22bitops-nsieve-bits%22:%5B126,127,127,138,130%5D,%22controlflow-recursive%22:%5B32,32,32,33,32%5D,%22crypto-aes%22:%5B74,72,75,71,73%5D,%22crypto-md5%22:%5B91,86,127,84,84%5D,%22crypto-sha1%22:%5B81,98,81,93,81%5D,%22date-format-tofte%22:%5B119,133,118,120,140%5D,%22date-format-xparb%22:%5B160,162,157,135,146%5D,%22math-cordic%22:%5B229,258,238,351,259%5D,%22math-partial-sums%22:%5B278,276,277,293,294%5D,%22math-spectral-norm%22:%5B135,187,171,153,146%5D,%22regexp-dna%22:%5B646,630,625,640,718%5D,%22string-base64%22:%5B92,91,94,95,93%5D,%22string-fasta%22:%5B149,173,152,159,171%5D,%22string-tagcloud%22:%5B185,204,195,172,172%5D,%22string-unpack-code%22:%5B261,258,265,284,262%5D,%22string-validate-input%22:%5B206,200,184,200,222%5D%7D


on a PowerBook G4 15" 1.67GHz 2GB RAM running Mac OS X 10.4.11

[Geoffrey Garen]

The URLs you pasted show now regression. What am I missing?

[Gavin Sherlock]

There are lots of things significant in those URLS:

binary-trees:      *1.83x as slow*     38.6ms +/- 3.7%     70.8ms +/- 16.3%     significant
bitwise-and:       *1.18x as slow*     71.0ms +/- 12.2%     83.6ms +/- 7.8%     significant
recursive:         *1.07x as slow*     32.2ms +/- 1.7%     34.4ms +/- 5.5%     significant
aes:               *1.18x as slow*     73.0ms +/- 2.7%     85.8ms +/- 12.1%     significant
format-xparb:      *1.61x as slow*    152.0ms +/- 9.3%    244.4ms +/- 19.2%     significant
tagcloud:          *1.90x as slow*    185.6ms +/- 9.5%    353.4ms +/- 12.9%     significant
validate-input:    *1.17x as slow*    202.4ms +/- 8.4%    236.0ms +/- 19.9%     significant

though the overall time is faster:

** TOTAL **:           -                 4751.2ms +/- 2.6%   4708.0ms +/- 3.5%

[Gavin Sherlock]

I also see some slow downs on Intel with 10.5.6.  Compare:

http://www2.webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B72,71,71,72,71%5D,%223d-morph%22:%5B57,53,54,54,53%5D,%223d-raytrace%22:%5B36,38,38,40,38%5D,%22access-binary-trees%22:%5B6,6,6,6,6%5D,%22access-fannkuch%22:%5B13,13,13,13,14%5D,%22access-nbody%22:%5B52,52,52,53,53%5D,%22access-nsieve%22:%5B7,7,7,7,8%5D,%22bitops-3bit-bits-in-byte%22:%5B4,3,3,3,3%5D,%22bitops-bits-in-byte%22:%5B6,6,7,7,6%5D,%22bitops-bitwise-and%22:%5B3,2,3,2,3%5D,%22bitops-nsieve-bits%22:%5B22,21,23,23,22%5D,%22controlflow-recursive%22:%5B3,4,4,4,4%5D,%22crypto-aes%22:%5B15,14,15,14,14%5D,%22crypto-md5%22:%5B21,21,22,21,20%5D,%22crypto-sha1%22:%5B19,19,20,19,19%5D,%22date-format-tofte%22:%5B31,32,31,31,32%5D,%22date-format-xparb%22:%5B37,37,39,38,38%5D,%22math-cordic%22:%5B45,46,47,46,45%5D,%22math-partial-sums%22:%5B46,45,46,45,46%5D,%22math-spectral-norm%22:%5B21,22,22,21,21%5D,%22regexp-dna%22:%5B31,32,32,32,32%5D,%22string-base64%22:%5B23,21,22,21,24%5D,%22string-fasta%22:%5B36,36,36,41,38%5D,%22string-tagcloud%22:%5B41,43,40,44,42%5D,%22string-unpack-code%22:%5B58,61,58,57,59%5D,%22string-validate-input%22:%5B43,42,41,43,45%5D%7D

to:

http://www2.webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B53,55,55,54,53%5D,%223d-morph%22:%5B53,52,52,53,53%5D,%223d-raytrace%22:%5B35,34,33,33,33%5D,%22access-binary-trees%22:%5B10,10,10,10,10%5D,%22access-fannkuch%22:%5B13,13,14,14,14%5D,%22access-nbody%22:%5B38,39,37,37,37%5D,%22access-nsieve%22:%5B7,8,10,7,7%5D,%22bitops-3bit-bits-in-byte%22:%5B3,3,4,4,3%5D,%22bitops-bits-in-byte%22:%5B6,6,7,6,7%5D,%22bitops-bitwise-and%22:%5B8,8,8,8,9%5D,%22bitops-nsieve-bits%22:%5B17,17,18,21,17%5D,%22controlflow-recursive%22:%5B5,4,4,4,5%5D,%22crypto-aes%22:%5B14,14,14,14,15%5D,%22crypto-md5%22:%5B17,17,17,18,17%5D,%22crypto-sha1%22:%5B16,17,16,16,16%5D,%22date-format-tofte%22:%5B30,29,29,30,31%5D,%22date-format-xparb%22:%5B45,44,45,45,47%5D,%22math-cordic%22:%5B36,37,36,44,35%5D,%22math-partial-sums%22:%5B49,48,49,49,48%5D,%22math-spectral-norm%22:%5B15,16,16,16,16%5D,%22regexp-dna%22:%5B30,31,30,30,31%5D,%22string-base64%22:%5B20,20,20,21,21%5D,%22string-fasta%22:%5B35,35,34,36,36%5D,%22string-tagcloud%22:%5B90,89,88,90,92%5D,%22string-unpack-code%22:%5B57,55,54,59,62%5D,%22string-validate-input%22:%5B51,51,58,54,58%5D%7D

r43444 is faster overall (just), but there are regressions in individual tests.

Some of the tests are now so quick, that there is a lot of variance.  It might be useful if Sunspider were updated for tests to take a little longer, and be run 5 times instead of 3.

[Gavin Barraclough]

I don't think anyone is actively working on PPC performance, and I don't think we'll still want to track individual regressions this old.  I think this is realistically won't fix.