68136 – Tiered compilation should be enabled by default on platforms that support the DFG JIT

RESOLVED FIXED68136

Tiered compilation should be enabled by default on platforms that support the DFG JIT

https://bugs.webkit.org/show_bug.cgi?id=68136

Summary Tiered compilation should be enabled by default on platforms that support the...

Filip Pizlo

Reported 2011-09-14 18:10:57 PDT

Tiered compilation is now a pure win on all benchmark, running both in the command-line bencher harness and when running in in-browser harnesses. It should be turned on by default. Performance results are below. Tests in command-line using bencher come first, then after that I show results in browser. Summary: SunSpider: no change V8: 2.6% speed-up in command line, 3% in browser Kraken: 6% speed-up in command line, 6.9% in browser. Benchmark report for SunSpider, V8, and Kraken. VMs tested: "TipOfTree" at /Volumes/Data/pizlo/secondary/OpenSource/WebKitBuild/Release/jsc "TipOfTreeDyn" at /Volumes/Data/pizlo/tertiary/OpenSource/WebKitBuild/Release/jsc Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark.Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in milliseconds. TipOfTree TipOfTreeDyn SunSpider: 3d-cube 8.4302+-0.0222 ^ 8.3198+-0.0266 ^ definitely 1.0133x faster 3d-morph 8.0814+-0.0199 8.0445+-0.0216 3d-raytrace 8.1582+-0.1157 8.1228+-0.0901 access-binary-trees 2.5265+-0.0284 ? 2.5576+-0.0348 ? might be 1.0123x slower access-fannkuch 12.8357+-0.0537 ? 12.8565+-0.1030 ? access-nbody 4.9249+-0.0131 ^ 4.8798+-0.0267 ^ definitely 1.0092x faster access-nsieve 3.0600+-0.0317 ! 3.1244+-0.0218 ! definitely 1.0211x slower bitops-3bit-bits-in-byte 1.8164+-0.0165 ^ 1.7234+-0.0227 ^ definitely 1.0539x faster bitops-bits-in-byte 4.5293+-0.0526 4.4825+-0.0408 might be 1.0104x faster bitops-bitwise-and 4.1277+-0.0398 4.1165+-0.0351 bitops-nsieve-bits 5.6712+-0.0435 ? 5.7023+-0.0300 ? controlflow-recursive 2.3055+-0.0273 2.2834+-0.0296 crypto-aes 6.9125+-0.0430 ! 7.2886+-0.1815 ! definitely 1.0544x slower crypto-md5 3.1190+-0.0591 3.0258+-0.0589 might be 1.0308x faster crypto-sha1 2.4611+-0.0318 2.4469+-0.0317 date-format-tofte 10.8462+-0.0381 ? 11.0204+-0.1420 ? might be 1.0161x slower date-format-xparb 9.4480+-0.0600 ^ 9.0535+-0.1051 ^ definitely 1.0436x faster math-cordic 7.1222+-0.0299 7.1005+-0.0217 math-partial-sums 10.7093+-0.0171 10.5327+-0.1899 might be 1.0168x faster math-spectral-norm 2.7595+-0.0303 ! 2.8293+-0.0251 ! definitely 1.0253x slower regexp-dna 12.6190+-0.1450 12.5731+-0.1250 string-base64 6.5514+-0.0865 ^ 6.1306+-0.0649 ^ definitely 1.0687x faster string-fasta 8.3384+-0.0543 ! 9.0432+-0.0120 ! definitely 1.0845x slower string-tagcloud 13.4682+-0.0862 13.3401+-0.0473 string-unpack-code 21.2893+-0.1294 ^ 20.8310+-0.1483 ^ definitely 1.0220x faster string-validate-input 7.4814+-0.2716 ^ 7.0612+-0.0435 ^ definitely 1.0595x faster <arithmetic> 7.2920+-0.0323 7.2496+-0.0274 <geometric> 6.0408+-0.0301 6.0064+-0.0286 <harmonic> 4.9374+-0.0297 4.8995+-0.0310 TipOfTree TipOfTreeDyn V8: crypto 103.2552+-0.4108 ^ 93.5304+-0.3761 ^ definitely 1.1040x faster deltablue 292.5504+-2.2242 ^ 285.5507+-2.4859 ^ definitely 1.0245x faster earley-boyer 112.3912+-0.2282 ! 114.2831+-0.4305 ! definitely 1.0168x slower raytrace 87.6601+-0.5881 86.8542+-0.2573 regexp 132.9250+-0.5788 ^ 130.0159+-0.5122 ^ definitely 1.0224x faster richards 300.6141+-1.8467 ^ 284.7736+-1.3857 ^ definitely 1.0556x faster splay 145.5287+-1.1222 ? 146.5276+-0.8034 ? <arithmetic> 167.8464+-0.4878 ^ 163.0765+-0.5047 ^ definitely 1.0292x faster <geometric> 150.2703+-0.3533 ^ 146.3499+-0.2865 ^ definitely 1.0268x faster <harmonic> 136.8999+-0.2936 ^ 133.3658+-0.1962 ^ definitely 1.0265x faster TipOfTree TipOfTreeDyn Kraken: ai-astar 1667.9173+-16.6919 ! 1710.0280+-18.8938 ! definitely 1.0252x slower audio-beat-detection 541.8131+-1.8633 ^ 533.0010+-1.7733 ^ definitely 1.0165x faster audio-dft 452.0701+-8.0429 ? 458.7987+-6.4440 ? might be 1.0149x slower audio-fft 421.3918+-0.8967 ^ 416.0566+-0.8896 ^ definitely 1.0128x faster audio-oscillator 403.1402+-0.6423 ^ 372.7753+-4.5984 ^ definitely 1.0815x faster imaging-darkroom 585.7165+-13.8667 ^ 467.8477+-1.8063 ^ definitely 1.2519x faster imaging-desaturate 652.0937+-0.1374 ^ 251.4218+-0.0225 ^ definitely 2.5936x faster imaging-gaussian-blur 1844.7490+-1.9134 1844.4333+-1.5254 json-parse-financial 63.1134+-0.3853 62.6635+-0.2933 json-stringify-tinderbox 83.7694+-0.3490 83.7252+-0.4869 stanford-crypto-aes 166.9100+-1.2094 ^ 164.1889+-0.4856 ^ definitely 1.0166x faster stanford-crypto-ccm 131.0104+-0.6111 ? 131.2150+-1.6770 ? stanford-crypto-pbkdf2 370.9821+-1.3787 ! 436.4344+-3.0435 ! definitely 1.1764x slower stanford-crypto-sha256-iterative 143.6550+-0.1511 ! 163.3373+-0.4790 ! definitely 1.1370x slower <arithmetic> 537.7380+-0.8751 ^ 506.8519+-1.8089 ^ definitely 1.0609x faster <geometric> 342.1178+-0.8278 ^ 319.0591+-0.9121 ^ definitely 1.0723x faster <harmonic> 217.5167+-0.4851 ^ 210.7909+-0.6950 ^ definitely 1.0319x faster TipOfTree TipOfTreeDyn All benchmarks: <arithmetic> 189.2096+-0.2348 ^ 179.2756+-0.5298 ^ definitely 1.0554x faster <geometric> 32.4465+-0.0987 ^ 31.5542+-0.0959 ^ definitely 1.0283x faster <harmonic> 8.7337+-0.0514 8.6626+-0.0538 Tests in browser: FROM = TipOfTree TO = Tiered Compilation TEST COMPARISON FROM TO DETAILS ============================================================================= ** TOTAL **: ?? 191.3ms +/- 1.0% 191.6ms +/- 0.6% not conclusive: might be *1.002x as slow* ============================================================================= 3d: - 25.1ms +/- 0.9% 25.0ms +/- 0.0% cube: - 9.0ms +/- 0.0% 9.0ms +/- 0.0% morph: - 8.0ms +/- 0.0% 8.0ms +/- 0.0% raytrace: - 8.1ms +/- 2.8% 8.0ms +/- 0.0% access: ?? 22.8ms +/- 3.2% 23.5ms +/- 2.6% not conclusive: might be *1.031x as slow* binary-trees: *1.21x as slow* 2.4ms +/- 15.4% 2.9ms +/- 7.8% significant fannkuch: - 12.8ms +/- 2.4% 12.8ms +/- 2.4% nbody: ?? 4.6ms +/- 8.0% 4.7ms +/- 7.3% not conclusive: might be *1.022x as slow* nsieve: ?? 3.0ms +/- 0.0% 3.1ms +/- 7.3% not conclusive: might be *1.033x as slow* bitops: ?? 15.5ms +/- 5.9% 16.0ms +/- 2.1% not conclusive: might be *1.032x as slow* 3bit-bits-in-byte: - 1.6ms +/- 23.1% 1.5ms +/- 25.1% bits-in-byte: ?? 4.4ms +/- 8.4% 4.5ms +/- 8.4% not conclusive: might be *1.023x as slow* bitwise-and: - 4.0ms +/- 0.0% 4.0ms +/- 0.0% nsieve-bits: *1.091x as slow* 5.5ms +/- 6.8% 6.0ms +/- 0.0% significant controlflow: ?? 2.0ms +/- 0.0% 2.1ms +/- 10.8% not conclusive: might be *1.050x as slow* recursive: ?? 2.0ms +/- 0.0% 2.1ms +/- 10.8% not conclusive: might be *1.050x as slow* crypto: *1.092x as slow* 12.0ms +/- 4.0% 13.1ms +/- 3.1% significant aes: *1.125x as slow* 7.2ms +/- 4.2% 8.1ms +/- 2.8% significant md5: ?? 2.6ms +/- 14.2% 2.9ms +/- 7.8% not conclusive: might be *1.115x as slow* sha1: - 2.2ms +/- 13.7% 2.1ms +/- 10.8% date: - 23.0ms +/- 2.1% 22.7ms +/- 1.5% format-tofte: ?? 12.8ms +/- 3.5% 12.9ms +/- 3.1% not conclusive: might be *1.008x as slow* format-xparb: 1.041x as fast 10.2ms +/- 3.0% 9.8ms +/- 3.1% significant math: - 20.5ms +/- 3.8% 20.2ms +/- 2.2% cordic: - 7.2ms +/- 6.3% 7.1ms +/- 3.2% partial-sums: - 10.6ms +/- 3.5% 10.3ms +/- 3.4% spectral-norm: ?? 2.7ms +/- 12.8% 2.8ms +/- 10.8% not conclusive: might be *1.037x as slow* regexp: - 11.9ms +/- 3.4% 11.4ms +/- 3.2% dna: - 11.9ms +/- 3.4% 11.4ms +/- 3.2% string: - 58.5ms +/- 1.9% 57.6ms +/- 2.0% base64: 1.167x as fast 7.7ms +/- 8.8% 6.6ms +/- 5.6% significant fasta: *1.059x as slow* 8.5ms +/- 4.4% 9.0ms +/- 0.0% significant tagcloud: ?? 12.6ms +/- 2.9% 12.9ms +/- 1.8% not conclusive: might be *1.024x as slow* unpack-code: ?? 21.3ms +/- 2.3% 21.4ms +/- 3.2% not conclusive: might be *1.005x as slow* validate-input: 1.091x as fast 8.4ms +/- 5.9% 7.7ms +/- 4.5% significant TipOfTree: Score: 4660 Richards: 4084 DeltaBlue: 3587 Crypto: 9954 RayTrace: 5228 EarleyBoyer: 8052 RegExp: 1787 Splay: 4352 TieredCompilation: (3% faster) Score: 4806 Richards: 4377 DeltaBlue: 3623 Crypto: 10748 RayTrace: 5416 EarleyBoyer: 7892 RegExp: 1785 Splay: 4555 FROM = TipOfTree TO = Tiered Compilation TEST COMPARISON FROM TO DETAILS ==================================================================================== ** TOTAL **: 1.069x as fast 7619.7ms +/- 0.2% 7129.9ms +/- 0.4% significant ==================================================================================== ai: ?? 1690.9ms +/- 1.1% 1717.7ms +/- 1.5% might be *1.016x as slow* astar: ?? 1690.9ms +/- 1.1% 1717.7ms +/- 1.5% might be *1.016x as slow* audio: 1.033x as fast 1842.6ms +/- 0.2% 1784.4ms +/- 0.3% significant beat-detection: 1.024x as fast 563.0ms +/- 0.5% 549.6ms +/- 0.7% significant dft: *1.009x as slow* 450.9ms +/- 0.3% 455.1ms +/- 0.7% significant fft: 1.016x as fast 425.6ms +/- 0.7% 419.0ms +/- 0.6% significant oscillator: 1.118x as fast 403.1ms +/- 0.4% 360.7ms +/- 0.4% significant imaging: 1.21x as fast 3095.3ms +/- 0.1% 2557.2ms +/- 0.2% significant gaussian-blur: 1.002x as fast 1819.6ms +/- 0.1% 1816.3ms +/- 0.1% significant darkroom: 1.27x as fast 622.6ms +/- 0.3% 488.9ms +/- 1.2% significant desaturate: 2.59x as fast 653.1ms +/- 0.2% 252.0ms +/- 0.4% significant json: - 159.1ms +/- 0.6% 159.1ms +/- 0.3% parse-financial: - 75.2ms +/- 1.1% 74.6ms +/- 0.5% stringify-tinderbox: *1.007x as slow* 83.9ms +/- 0.5% 84.5ms +/- 0.4% significant stanford: *1.096x as slow* 831.8ms +/- 0.5% 911.5ms +/- 0.3% significant crypto-aes: ?? 166.2ms +/- 1.3% 167.4ms +/- 1.5% might be *1.007x as slow* crypto-ccm: - 142.5ms +/- 1.1% 142.0ms +/- 0.8% crypto-pbkdf2: *1.167x as slow* 373.3ms +/- 0.8% 435.8ms +/- 0.7% significant crypto-sha256-iterative: *1.110x as slow* 149.8ms +/- 0.7% 166.3ms +/- 0.7% significant

Attachments
the patch (1.27 KB, patch) 2011-09-14 18:18 PDT, Filip Pizlo	webkit-ews: commit-queue-	Details Formatted Diff Diff
the patch - fix everything (1.34 KB, patch) 2011-09-14 18:39 PDT, Filip Pizlo	webkit-ews: commit-queue-	Details Formatted Diff Diff
the patch - for real this time (2.09 KB, patch) 2011-09-14 19:05 PDT, Filip Pizlo	sam: review+	Details Formatted Diff Diff
Show Obsolete (2) View All Add attachment proposed patch, testcase, etc.

Filip Pizlo

Comment 1 2011-09-14 18:18:55 PDT

Created attachment 107435 [details] the patch

Early Warning System Bot

Comment 2 2011-09-14 18:36:43 PDT

Comment on attachment 107435 [details] the patch Attachment 107435 [details] did not pass qt-ews (qt): Output: http://queues.webkit.org/results/9661764

Filip Pizlo

Comment 3 2011-09-14 18:39:04 PDT

Created attachment 107437 [details] the patch - fix everything

Early Warning System Bot

Comment 4 2011-09-14 18:51:18 PDT

Comment on attachment 107437 [details] the patch - fix everything Attachment 107437 [details] did not pass qt-ews (qt): Output: http://queues.webkit.org/results/9655784

Filip Pizlo

Comment 5 2011-09-14 19:05:04 PDT

Created attachment 107442 [details] the patch - for real this time

Filip Pizlo

Comment 6 2011-09-14 20:42:14 PDT

Will wait for the bots to go green before committing, since I'm not yet 100% sure that I've got all those #ifdef's right.

Filip Pizlo

Comment 7 2011-09-15 11:54:38 PDT

Landed in r95206. The last numbers (comparing 95205 to 95206): Benchmark report for SunSpider, V8, and Kraken. VMs tested: "TipOfTree" at /Volumes/Data/pizlo/octonary/OpenSource/WebKitBuild/Release/jsc "TieredCompilation" at /Volumes/Data/pizlo/quinary/OpenSource/WebKitBuild/Release/jsc Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in milliseconds. TipOfTree TieredCompilation SunSpider: 3d-cube 7.9594+-0.2150 7.9262+-0.1658 3d-morph 7.5548+-0.1317 ? 7.5557+-0.1786 ? 3d-raytrace 8.2024+-0.3451 7.6801+-0.2302 might be 1.0680x faster access-binary-trees 2.2097+-0.0229 ! 2.2953+-0.0507 ! definitely 1.0387x slower access-fannkuch 11.6981+-0.2218 ? 11.8111+-0.2361 ? access-nbody 4.3410+-0.1316 4.2497+-0.1391 might be 1.0215x faster access-nsieve 2.5923+-0.0997 ? 2.6411+-0.0757 ? might be 1.0188x slower bitops-3bit-bits-in-byte 1.7042+-0.0569 ? 1.7102+-0.0685 ? bitops-bits-in-byte 2.7652+-0.0643 ? 2.8248+-0.1128 ? might be 1.0215x slower bitops-bitwise-and 3.7014+-0.1329 3.6661+-0.1160 bitops-nsieve-bits 5.4193+-0.0787 5.2836+-0.1220 might be 1.0257x faster controlflow-recursive 1.9813+-0.0341 ? 2.0904+-0.0850 ? might be 1.0551x slower crypto-aes 6.7605+-0.2467 ? 7.1447+-0.3279 ? might be 1.0568x slower crypto-md5 2.7752+-0.0483 ? 3.3646+-0.6840 ? might be 1.2124x slower crypto-sha1 2.2489+-0.0621 2.1552+-0.0704 might be 1.0435x faster date-format-tofte 11.2852+-0.3376 ^ 10.3442+-0.2719 ^ definitely 1.0910x faster date-format-xparb 8.8027+-0.2859 8.7357+-0.2227 math-cordic 6.3806+-0.1088 6.2527+-0.1303 might be 1.0204x faster math-partial-sums 7.6559+-0.1588 7.3674+-0.1612 might be 1.0392x faster math-spectral-norm 2.4454+-0.0401 ! 2.5832+-0.0443 ! definitely 1.0563x slower regexp-dna 11.1329+-0.2861 10.7977+-0.2008 might be 1.0310x faster string-base64 5.8202+-0.1199 5.8175+-0.1956 string-fasta 7.1207+-0.0827 ! 8.5540+-0.2553 ! definitely 1.2013x slower string-tagcloud 11.8230+-0.2290 ? 11.8602+-0.2660 ? string-unpack-code 19.1518+-0.4605 19.1024+-0.4931 string-validate-input 7.5095+-0.3377 ^ 6.6418+-0.1659 ^ definitely 1.1307x faster <arithmetic> 6.5785+-0.0437 6.5560+-0.0377 <geometric> 5.3869+-0.0303 ? 5.4105+-0.0375 ? <harmonic> 4.3522+-0.0249 ? 4.4074+-0.0392 ? might be 1.0127x slower TipOfTree TieredCompilation V8: crypto 90.8545+-0.8925 ^ 85.2666+-2.3793 ^ definitely 1.0655x faster deltablue 265.8815+-3.5027 ^ 247.8677+-2.3100 ^ definitely 1.0727x faster earley-boyer 94.4787+-0.7301 ? 95.5216+-0.8552 ? might be 1.0110x slower raytrace 76.1163+-1.1740 ^ 71.9430+-0.4490 ^ definitely 1.0580x faster regexp 108.4889+-1.3826 107.0564+-0.5068 might be 1.0134x faster richards 239.9382+-2.8300 ^ 220.9082+-2.7744 ^ definitely 1.0861x faster splay 99.3576+-0.2353 ? 99.8593+-0.4493 ? <arithmetic> 139.3022+-0.6465 ^ 132.6318+-0.5601 ^ definitely 1.0503x faster <geometric> 123.9271+-0.5100 ^ 119.2335+-0.4920 ^ definitely 1.0394x faster <harmonic> 112.9873+-0.5110 ^ 109.3138+-0.5171 ^ definitely 1.0336x faster TipOfTree TieredCompilation Kraken: ai-astar 1220.2982+-33.6219 ^ 651.2664+-13.5760 ^ definitely 1.8737x faster audio-beat-detection 481.6183+-3.7772 476.8139+-5.5267 might be 1.0101x faster audio-dft 423.7553+-5.9952 417.3201+-3.1266 might be 1.0154x faster audio-fft 369.7725+-0.7480 369.4925+-4.5536 audio-oscillator 392.0498+-16.2092 ^ 323.1123+-7.5327 ^ definitely 1.2134x faster imaging-darkroom 538.4024+-9.1255 ^ 417.7482+-3.7546 ^ definitely 1.2888x faster imaging-desaturate 629.2546+-14.4686 ^ 211.4487+-1.9027 ^ definitely 2.9759x faster imaging-gaussian-blur 1729.7578+-6.4878 ? 1739.8427+-19.4150 ? json-parse-financial 48.4246+-0.4196 ? 49.0291+-0.8502 ? might be 1.0125x slower json-stringify-tinderbox 70.2918+-2.9485 68.1125+-0.4787 might be 1.0320x faster stanford-crypto-aes 147.8755+-2.6111 ^ 144.7200+-0.4577 ^ definitely 1.0218x faster stanford-crypto-ccm 113.2348+-1.2837 111.8843+-0.7783 might be 1.0121x faster stanford-crypto-pbkdf2 339.8100+-9.8956 ! 404.5472+-2.2667 ! definitely 1.1905x slower stanford-crypto-sha256-iterative 132.5923+-0.9317 ! 150.5893+-1.3005 ! definitely 1.1357x slower <arithmetic> 474.0813+-3.5488 ^ 395.4234+-1.9613 ^ definitely 1.1989x faster <geometric> 302.5975+-1.5819 ^ 263.5863+-0.6790 ^ definitely 1.1480x faster <harmonic> 185.2691+-1.1003 ^ 175.9015+-0.5961 ^ definitely 1.0533x faster TipOfTree TieredCompilation All benchmarks: <arithmetic> 165.6020+-1.0020 ^ 141.1661+-0.5955 ^ definitely 1.1731x faster <geometric> 28.5295+-0.1039 ^ 27.2891+-0.1128 ^ definitely 1.0455x faster <harmonic> 7.6904+-0.0429 ? 7.7777+-0.0675 ? might be 1.0114x slower

Note You need to log in before you can comment on or make changes to this bug.

Status RESOLVED

Resolution FIXED

Priority P2

Severity Normal

Classification Unclassified

Version 528+ (Nightly build)

Hardware All

OS All

Product WebKit

Component JavaScriptCore

Assignee

Nobody

Reported

2011-09-14 18:10 PDT

Modified

2011-09-15 11:54 PDT History

CC List

0 users

URL

Keywords

Depends on

Blocks