Bug 73236
| Summary: | Slowdown in Kraken possibly due to 8 bit String work | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Michael Saboff <msaboff> |
| Component: | JavaScriptCore | Assignee: | Michael Saboff <msaboff> |
| Status: | RESOLVED FIXED | ||
| Severity: | Normal | CC: | ggaren |
| Priority: | P2 | Keywords: | InRadar |
| Version: | 528+ (Nightly build) | ||
| Hardware: | All | ||
| OS: | All | ||
| Bug Depends on: | 73457, 73584, 73871, 73875, 73882 | ||
| Bug Blocks: | |||
Michael Saboff
From <rdar://problem/10491903>
It appears that several Kraken tests are slower due to 8 bit String work. The tests are json-parse-financial, json-stringify-tinderbox, stanford-crypto-aes, stanford-crypto-ccm, stanford-crypto-pbkdf2 and stanford-crypto-sha256-iterative.
Test results using bencher comparing before (r100509) and after (r101201):
Sending VM builds to bigmac...
Running on bigmac...
376/376
Generating benchmark report at Before8Bit_After8Bit_SunSpiderV8Kraken_20111127_0929_benchReport.txt
Benchmark report for SunSpider, V8, and Kraken on bigmac.local (MacPro5,1).
VMs tested:
"Before8Bit" at /Volumes/Data/pizlo/senary/OpenSource/WebKitBuild/Release/jsc (r100509)
"After8Bit" at /Volumes/Data/pizlo/quinary/OpenSource/WebKitBuild/Release/jsc (r101201)
Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between sample
measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime()
function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in
milliseconds.
Before8Bit After8Bit
SunSpider:
3d-cube 7.4983+-0.0202 ? 7.5172+-0.0205 ?
3d-morph 8.4660+-0.0918 8.3753+-0.0388 might be 1.0108x faster
3d-raytrace 7.6919+-0.0593 ? 7.6935+-0.0665 ?
access-binary-trees 1.5993+-0.0032 ? 1.6101+-0.0173 ?
access-fannkuch 7.5880+-0.0620 7.5261+-0.0106
access-nbody 4.1796+-0.0070 4.1744+-0.0111
access-nsieve 3.1496+-0.0402 3.1451+-0.0402
bitops-3bit-bits-in-byte 1.2392+-0.0135 1.2335+-0.0057
bitops-bits-in-byte 4.9073+-0.0072 ? 4.9165+-0.0119 ?
bitops-bitwise-and 3.2899+-0.0045 ? 3.3321+-0.0412 ? might be 1.0128x slower
bitops-nsieve-bits 5.6804+-0.0330 5.6640+-0.0352
controlflow-recursive 2.2966+-0.0234 ? 2.2968+-0.0249 ?
crypto-aes 7.1320+-0.0361 ? 7.2682+-0.1182 ? might be 1.0191x slower
crypto-md5 2.5062+-0.0146 2.4988+-0.0284
crypto-sha1 2.1845+-0.0240 2.1777+-0.0256
date-format-tofte 10.6530+-0.0579 ? 10.7872+-0.1459 ? might be 1.0126x slower
date-format-xparb 10.9905+-0.1565 10.7874+-0.2014 might be 1.0188x faster
math-cordic 7.1240+-0.0177 ? 7.2002+-0.0856 ? might be 1.0107x slower
math-partial-sums 10.4612+-0.0248 10.4400+-0.0131
math-spectral-norm 2.5962+-0.0078 ? 2.6074+-0.0165 ?
regexp-dna 13.3500+-0.1339 ^ 12.9810+-0.0571 ^ definitely 1.0284x faster
string-base64 3.9595+-0.0256 3.9230+-0.0114
string-fasta 7.0986+-0.0498 ! 7.3195+-0.0244 ! definitely 1.0311x slower
string-tagcloud 13.3659+-0.0782 ^ 12.4848+-0.0581 ^ definitely 1.0706x faster
string-unpack-code 22.7048+-0.0776 ^ 22.3886+-0.0952 ^ definitely 1.0141x faster
string-validate-input 5.6269+-0.0165 5.6224+-0.0333
<arithmetic> * 6.8207+-0.0205 ^ 6.7681+-0.0215 ^ definitely 1.0078x faster
<geometric> 5.4093+-0.0148 5.3935+-0.0186 might be 1.0029x faster
<harmonic> 4.2003+-0.0116 4.1985+-0.0168 might be 1.0004x faster
Before8Bit After8Bit
V8:
crypto 77.1493+-0.3012 ? 77.4807+-0.3348 ?
deltablue 172.2109+-0.7602 ^ 169.7490+-0.6978 ^ definitely 1.0145x faster
earley-boyer 105.0439+-1.3705 104.0297+-0.6661
raytrace 62.6072+-0.3002 ? 63.0565+-0.4008 ?
regexp 126.2524+-0.7232 ^ 123.5293+-0.6461 ^ definitely 1.0220x faster
richards 138.5381+-1.6642 137.7148+-0.2749
splay 91.6718+-1.1884 91.5304+-1.1633
<arithmetic> 110.4962+-0.2612 ^ 109.5843+-0.2293 ^ definitely 1.0083x faster
<geometric> * 104.9257+-0.2594 ^ 104.3027+-0.2435 ^ definitely 1.0060x faster
<harmonic> 99.5454+-0.2595 99.2099+-0.2592 might be 1.0034x faster
Before8Bit After8Bit
Kraken:
ai-astar 807.8610+-12.9121 ? 817.0948+-10.7504 ? might be 1.0114x slower
audio-beat-detection 205.3480+-0.8559 205.0317+-0.3020
audio-dft 263.2363+-2.0675 261.0770+-2.1302
audio-fft 132.8394+-0.0992 ? 133.4540+-0.8572 ?
audio-oscillator 292.7988+-0.4909 ^ 282.7311+-8.0606 ^ definitely 1.0356x faster
imaging-darkroom 338.0414+-5.9347 333.0888+-5.1154 might be 1.0149x faster
imaging-desaturate 240.8554+-0.1107 ^ 238.5878+-0.0806 ^ definitely 1.0095x faster
imaging-gaussian-blur 620.6508+-0.2048 620.4488+-0.1733
json-parse-financial 71.6349+-0.2139 ! 73.6682+-0.0914 ! definitely 1.0284x slower
json-stringify-tinderbox 79.9620+-0.2514 ! 85.7804+-0.2587 ! definitely 1.0728x slower
stanford-crypto-aes 116.9000+-0.6996 ! 119.4791+-1.3637 ! definitely 1.0221x slower
stanford-crypto-ccm 116.3499+-0.7296 ! 118.0601+-0.7966 ! definitely 1.0147x slower
stanford-crypto-pbkdf2 231.0752+-0.5542 ! 235.5944+-1.3989 ! definitely 1.0196x slower
stanford-crypto-sha256-iterative 97.4963+-0.3098 ? 97.8114+-0.2592 ?
<arithmetic> * 258.2178+-1.0382 ? 258.7077+-0.9749 ? might be 1.0019x slower
<geometric> 198.7937+-0.4374 ! 200.2622+-0.5083 ! definitely 1.0074x slower
<harmonic> 159.4982+-0.2785 ! 162.1874+-0.3615 ! definitely 1.0169x slower
Before8Bit After8Bit
All benchmarks:
<arithmetic> 97.1460+-0.3238 97.1270+-0.2962 might be 1.0002x faster
<geometric> 24.6137+-0.0434 24.6062+-0.0638 might be 1.0003x faster
<harmonic> 7.4037+-0.0201 7.4021+-0.0292 might be 1.0002x faster
Before8Bit After8Bit
Geomean of preferred means:
<scaled-result> 56.9591+-0.1094 56.7355+-0.1267 might be 1.0039x faster
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Michael Saboff
With the changes checked in to the dependent bugs, performance for Kraken is now faster than in 100509 and 101201.
Marking defect as fixed.
376/376
Generating benchmark report at SunSpiderV8Kraken_20111208_1423_benchReport.txt
Benchmark report for SunSpider, V8, and Kraken on msaboff-pro.apple.com (MacPro5,1).
VMs tested:
"Conf#1" at /Volumes/Data/src/webkit.baseline/WebKitBuild/Release/jsc (r100509)
"Conf#2" at /Volumes/Data/src/webkit/WebKitBuild/Release/jsc (r102379)
Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between sample
measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime()
function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in
milliseconds.
Conf#1 Conf#2
SunSpider:
3d-cube 7.5586+-0.0876 ^ 7.3433+-0.0753 ^ definitely 1.0293x faster
3d-morph 8.6088+-0.1269 ^ 8.3779+-0.0426 ^ definitely 1.0276x faster
3d-raytrace 7.7935+-0.0676 7.7773+-0.0935
access-binary-trees 1.6113+-0.0155 1.6048+-0.0181
access-fannkuch 7.6050+-0.0556 7.5162+-0.0733 might be 1.0118x faster
access-nbody 4.1831+-0.0134 ^ 3.9090+-0.0087 ^ definitely 1.0701x faster
access-nsieve 3.1946+-0.0576 3.1637+-0.0458
bitops-3bit-bits-in-byte 1.2421+-0.0162 ? 1.2591+-0.0139 ? might be 1.0137x slower
bitops-bits-in-byte 4.9506+-0.0512 4.9464+-0.0540
bitops-bitwise-and 3.3219+-0.0355 3.3123+-0.0349
bitops-nsieve-bits 5.6616+-0.0363 5.6542+-0.0465
controlflow-recursive 2.2945+-0.0247 ? 2.3408+-0.0309 ? might be 1.0202x slower
crypto-aes 7.2325+-0.0783 ? 7.2595+-0.0671 ?
crypto-md5 2.5550+-0.0390 ^ 2.4541+-0.0288 ^ definitely 1.0411x faster
crypto-sha1 2.1819+-0.0273 2.1744+-0.0200
date-format-tofte 10.9461+-0.1771 10.7186+-0.1239 might be 1.0212x faster
date-format-xparb 11.0498+-0.1970 ^ 9.7705+-0.0980 ^ definitely 1.1309x faster
math-cordic 7.2203+-0.0787 7.1414+-0.0502 might be 1.0111x faster
math-partial-sums 10.5581+-0.0648 10.5407+-0.0385
math-spectral-norm 2.6093+-0.0252 ? 2.6162+-0.0265 ?
regexp-dna 13.4301+-0.1804 ^ 13.0621+-0.0907 ^ definitely 1.0282x faster
string-base64 3.9833+-0.0499 3.9599+-0.0113
string-fasta 7.1143+-0.0587 ! 7.3011+-0.0605 ! definitely 1.0263x slower
string-tagcloud 13.4213+-0.1133 ^ 12.4883+-0.0829 ^ definitely 1.0747x faster
string-unpack-code 22.7301+-0.1703 ^ 21.0415+-0.1434 ^ definitely 1.0802x faster
string-validate-input 5.6165+-0.0402 ? 5.6589+-0.0816 ?
<arithmetic> * 6.8721+-0.0258 ^ 6.6689+-0.0154 ^ definitely 1.0305x faster
<geometric> 5.4483+-0.0203 ^ 5.3516+-0.0102 ^ definitely 1.0181x faster
<harmonic> 4.2262+-0.0189 ^ 4.1906+-0.0072 ^ definitely 1.0085x faster
Conf#1 Conf#2
V8:
crypto 77.4807+-0.1983 76.9527+-0.3481
deltablue 172.4467+-1.7856 171.1052+-1.0379
earley-boyer 104.8629+-1.2395 ^ 99.6271+-1.0331 ^ definitely 1.0526x faster
raytrace 62.3679+-0.3792 ^ 57.7748+-0.2657 ^ definitely 1.0795x faster
regexp 125.4245+-0.3902 ^ 124.1845+-0.4750 ^ definitely 1.0100x faster
richards 137.6683+-0.5467 ! 139.9030+-1.3107 ! definitely 1.0162x slower
splay 90.7245+-0.9326 ? 91.6808+-1.2187 ? might be 1.0105x slower
<arithmetic> 110.1394+-0.4058 ^ 108.7469+-0.3685 ^ definitely 1.0128x faster
<geometric> * 104.5820+-0.3241 ^ 102.7199+-0.3235 ^ definitely 1.0181x faster
<harmonic> 99.2311+-0.2715 ^ 96.8161+-0.2870 ^ definitely 1.0249x faster
Conf#1 Conf#2
Kraken:
ai-astar 808.0324+-12.5951 ? 821.6116+-11.3976 ? might be 1.0168x slower
audio-beat-detection 206.3003+-1.0255 ^ 203.3832+-0.5119 ^ definitely 1.0143x faster
audio-dft 264.4750+-2.1162 261.3297+-2.0749 might be 1.0120x faster
audio-fft 133.6044+-0.6229 ^ 131.4506+-0.1236 ^ definitely 1.0164x faster
audio-oscillator 294.2943+-0.4086 ^ 278.9171+-5.3571 ^ definitely 1.0551x faster
imaging-darkroom 337.5491+-5.8683 333.4703+-4.5430 might be 1.0122x faster
imaging-desaturate 241.0463+-0.1571 ^ 239.1154+-0.2125 ^ definitely 1.0081x faster
imaging-gaussian-blur 621.8833+-0.3673 ^ 610.6176+-0.2940 ^ definitely 1.0184x faster
json-parse-financial 72.8350+-0.1946 ^ 71.1272+-0.2061 ^ definitely 1.0240x faster
json-stringify-tinderbox 80.6159+-0.3248 ! 81.6073+-0.2421 ! definitely 1.0123x slower
stanford-crypto-aes 117.4781+-0.4334 ! 118.7111+-0.5430 ! definitely 1.0105x slower
stanford-crypto-ccm 116.3493+-0.6721 ? 116.4044+-1.7463 ?
stanford-crypto-pbkdf2 232.6358+-1.6932 231.5536+-0.4345
stanford-crypto-sha256-iterative 97.2901+-0.1270 ^ 93.7459+-0.2283 ^ definitely 1.0378x faster
<arithmetic> * 258.8850+-0.9800 ^ 256.6461+-0.8476 ^ definitely 1.0087x faster
<geometric> 199.5898+-0.4201 ^ 197.3124+-0.3636 ^ definitely 1.0115x faster
<harmonic> 160.3821+-0.2812 ^ 158.5589+-0.3025 ^ definitely 1.0115x faster
Conf#1 Conf#2
All benchmarks:
<arithmetic> 97.3200+-0.2631 ^ 96.3333+-0.2482 ^ definitely 1.0102x faster
<geometric> 24.7291+-0.0579 ^ 24.3366+-0.0427 ^ definitely 1.0161x faster
<harmonic> 7.4486+-0.0326 ^ 7.3842+-0.0126 ^ definitely 1.0087x faster
Conf#1 Conf#2
Geomean of preferred means:
<scaled-result> 57.0882+-0.1130 ^ 56.0203+-0.1107 ^ definitely 1.0191x faster
And r101201 to 102379:
VMs tested:
"Conf#1" at /Volumes/Data/src/webkit.baseline/WebKitBuild/Release/jsc (r101201)
"Conf#2" at /Volumes/Data/src/webkit/WebKitBuild/Release/jsc (r102379)
Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between sample
measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime()
function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in
milliseconds.
Conf#1 Conf#2
SunSpider:
3d-cube 7.5249+-0.0654 ^ 7.3181+-0.0653 ^ definitely 1.0283x faster
3d-morph 8.4156+-0.0648 ? 8.5380+-0.1723 ? might be 1.0145x slower
3d-raytrace 7.7583+-0.0636 7.7391+-0.0871
access-binary-trees 1.5908+-0.0064 ? 1.6034+-0.0135 ?
access-fannkuch 7.5568+-0.0450 ^ 7.4472+-0.0321 ^ definitely 1.0147x faster
access-nbody 4.1942+-0.0303 ^ 3.9331+-0.0151 ^ definitely 1.0664x faster
access-nsieve 3.1568+-0.0464 ? 3.1682+-0.0535 ?
bitops-3bit-bits-in-byte 1.2378+-0.0124 ? 1.2421+-0.0118 ?
bitops-bits-in-byte 4.9308+-0.0449 ? 4.9483+-0.0458 ?
bitops-bitwise-and 3.2870+-0.0047 ? 3.2926+-0.0133 ?
bitops-nsieve-bits 5.6518+-0.0416 ? 5.6705+-0.0475 ?
controlflow-recursive 2.2987+-0.0260 ? 2.3113+-0.0248 ?
crypto-aes 7.1938+-0.0697 ! 7.3673+-0.0580 ! definitely 1.0241x slower
crypto-md5 2.5197+-0.0410 ^ 2.4271+-0.0125 ^ definitely 1.0381x faster
crypto-sha1 2.1652+-0.0201 ? 2.1686+-0.0231 ?
date-format-tofte 10.9027+-0.1206 ? 11.1582+-0.1398 ? might be 1.0234x slower
date-format-xparb 11.0719+-0.1033 ^ 10.0042+-0.1683 ^ definitely 1.1067x faster
math-cordic 7.2016+-0.0693 ? 7.2526+-0.0444 ?
math-partial-sums 10.4513+-0.0352 ? 10.5261+-0.0610 ?
math-spectral-norm 2.6060+-0.0179 ? 2.6446+-0.0368 ? might be 1.0148x slower
regexp-dna 13.0620+-0.0966 13.0337+-0.0677
string-base64 3.9888+-0.0422 ? 4.0174+-0.0795 ?
string-fasta 7.3779+-0.0625 7.3458+-0.0601
string-tagcloud 12.7663+-0.0868 ^ 12.5234+-0.0917 ^ definitely 1.0194x faster
string-unpack-code 22.4449+-0.1818 ^ 21.0416+-0.1830 ^ definitely 1.0667x faster
string-validate-input 5.5934+-0.0596 ! 5.7370+-0.0814 ! definitely 1.0257x slower
<arithmetic> * 6.8057+-0.0236 ^ 6.7100+-0.0206 ^ definitely 1.0143x faster
<geometric> 5.4114+-0.0203 ^ 5.3728+-0.0163 ^ definitely 1.0072x faster
<harmonic> 4.2023+-0.0196 4.1901+-0.0136
Conf#1 Conf#2
V8:
crypto 77.2326+-0.2614 76.7419+-0.3705
deltablue 173.2880+-1.3225 ^ 167.1205+-0.8145 ^ definitely 1.0369x faster
earley-boyer 104.6623+-1.0752 ^ 100.0545+-1.0791 ^ definitely 1.0461x faster
raytrace 62.7820+-0.4696 ^ 57.7076+-0.4545 ^ definitely 1.0879x faster
regexp 123.6062+-0.7665 ? 124.2000+-0.5179 ?
richards 137.9127+-0.5598 ? 138.6851+-1.3535 ?
splay 90.1932+-0.6703 ? 91.5607+-1.2582 ? might be 1.0152x slower
<arithmetic> 109.9539+-0.3874 ^ 108.0101+-0.4073 ^ definitely 1.0180x faster
<geometric> * 104.3993+-0.3431 ^ 102.2347+-0.4033 ^ definitely 1.0212x faster
<harmonic> 99.0999+-0.3251 ^ 96.5101+-0.3987 ^ definitely 1.0268x faster
Conf#1 Conf#2
Kraken:
ai-astar 819.6935+-11.2916 ? 820.7692+-11.4151 ?
audio-beat-detection 206.0952+-0.6926 ^ 203.2165+-0.3919 ^ definitely 1.0142x faster
audio-dft 267.5451+-2.4847 263.8061+-2.9047 might be 1.0142x faster
audio-fft 133.6997+-0.5249 ^ 131.6596+-0.2814 ^ definitely 1.0155x faster
audio-oscillator 282.2164+-6.0819 281.6886+-5.7926
imaging-darkroom 336.9450+-6.3205 331.7849+-4.3692 might be 1.0156x faster
imaging-desaturate 239.3408+-0.4562 ? 239.6299+-0.3858 ?
imaging-gaussian-blur 621.6622+-0.4815 ^ 611.1973+-0.6619 ^ definitely 1.0171x faster
json-parse-financial 73.8714+-0.2115 ^ 71.1441+-0.3116 ^ definitely 1.0383x faster
json-stringify-tinderbox 86.8911+-0.5000 ^ 81.8417+-0.1734 ^ definitely 1.0617x faster
stanford-crypto-aes 118.6045+-0.3383 ? 118.7973+-0.4983 ?
stanford-crypto-ccm 117.4189+-0.8437 116.0102+-1.5232 might be 1.0121x faster
stanford-crypto-pbkdf2 236.2737+-1.1218 ^ 231.6674+-0.6085 ^ definitely 1.0199x faster
stanford-crypto-sha256-iterative 97.0476+-0.2706 ^ 93.5726+-0.2026 ^ definitely 1.0371x faster
<arithmetic> * 259.8075+-1.1077 ^ 256.9132+-0.9048 ^ definitely 1.0113x faster
<geometric> 200.9451+-0.5411 ^ 197.5400+-0.4636 ^ definitely 1.0172x faster
<harmonic> 162.5505+-0.3069 ^ 158.6870+-0.3669 ^ definitely 1.0243x faster
Conf#1 Conf#2
All benchmarks:
<arithmetic> 97.5305+-0.3240 ^ 96.3259+-0.2767 ^ definitely 1.0125x faster
<geometric> 24.6798+-0.0684 ^ 24.3812+-0.0630 ^ definitely 1.0122x faster
<harmonic> 7.4087+-0.0339 7.3832+-0.0239
Conf#1 Conf#2
Geomean of preferred means:
<scaled-result> 56.9382+-0.1295 ^ 56.0659+-0.1369 ^ definitely 1.0156x faster