Bug 68136

Summary:

Tiered compilation should be enabled by default on platforms that support the DFG JIT

Product:

WebKit

Reporter:

Filip Pizlo <fpizlo>

Component:

JavaScriptCore

Assignee:

Nobody <webkit-unassigned>

Status:

RESOLVED FIXED

Severity:

Normal

Priority:

Version:

528+ (Nightly build)

Hardware:

All

OS:

All

Attachments:

Description	Flags
the patch	webkit-ews: commit-queue-
the patch - fix everything	webkit-ews: commit-queue-
the patch - for real this time	sam: review+

Filip Pizlo

Reported 2011-09-14 18:10:57 PDT

Tiered compilation is now a pure win on all benchmark, running both in the command-line bencher harness and when running in in-browser harnesses. It should be turned on by default. Performance results are below. Tests in command-line using bencher come first, then after that I show results in browser. Summary: SunSpider: no change V8: 2.6% speed-up in command line, 3% in browser Kraken: 6% speed-up in command line, 6.9% in browser. Benchmark report for SunSpider, V8, and Kraken. VMs tested: "TipOfTree" at /Volumes/Data/pizlo/secondary/OpenSource/WebKitBuild/Release/jsc "TipOfTreeDyn" at /Volumes/Data/pizlo/tertiary/OpenSource/WebKitBuild/Release/jsc Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark.Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in milliseconds. TipOfTree TipOfTreeDyn SunSpider: 3d-cube 8.4302+-0.0222 ^ 8.3198+-0.0266 ^ definitely 1.0133x faster 3d-morph 8.0814+-0.0199 8.0445+-0.0216 3d-raytrace 8.1582+-0.1157 8.1228+-0.0901 access-binary-trees 2.5265+-0.0284 ? 2.5576+-0.0348 ? might be 1.0123x slower access-fannkuch 12.8357+-0.0537 ? 12.8565+-0.1030 ? access-nbody 4.9249+-0.0131 ^ 4.8798+-0.0267 ^ definitely 1.0092x faster access-nsieve 3.0600+-0.0317 ! 3.1244+-0.0218 ! definitely 1.0211x slower bitops-3bit-bits-in-byte 1.8164+-0.0165 ^ 1.7234+-0.0227 ^ definitely 1.0539x faster bitops-bits-in-byte 4.5293+-0.0526 4.4825+-0.0408 might be 1.0104x faster bitops-bitwise-and 4.1277+-0.0398 4.1165+-0.0351 bitops-nsieve-bits 5.6712+-0.0435 ? 5.7023+-0.0300 ? controlflow-recursive 2.3055+-0.0273 2.2834+-0.0296 crypto-aes 6.9125+-0.0430 ! 7.2886+-0.1815 ! definitely 1.0544x slower crypto-md5 3.1190+-0.0591 3.0258+-0.0589 might be 1.0308x faster crypto-sha1 2.4611+-0.0318 2.4469+-0.0317 date-format-tofte 10.8462+-0.0381 ? 11.0204+-0.1420 ? might be 1.0161x slower date-format-xparb 9.4480+-0.0600 ^ 9.0535+-0.1051 ^ definitely 1.0436x faster math-cordic 7.1222+-0.0299 7.1005+-0.0217 math-partial-sums 10.7093+-0.0171 10.5327+-0.1899 might be 1.0168x faster math-spectral-norm 2.7595+-0.0303 ! 2.8293+-0.0251 ! definitely 1.0253x slower regexp-dna 12.6190+-0.1450 12.5731+-0.1250 string-base64 6.5514+-0.0865 ^ 6.1306+-0.0649 ^ definitely 1.0687x faster string-fasta 8.3384+-0.0543 ! 9.0432+-0.0120 ! definitely 1.0845x slower string-tagcloud 13.4682+-0.0862 13.3401+-0.0473 string-unpack-code 21.2893+-0.1294 ^ 20.8310+-0.1483 ^ definitely 1.0220x faster string-validate-input 7.4814+-0.2716 ^ 7.0612+-0.0435 ^ definitely 1.0595x faster <arithmetic> 7.2920+-0.0323 7.2496+-0.0274 <geometric> 6.0408+-0.0301 6.0064+-0.0286 <harmonic> 4.9374+-0.0297 4.8995+-0.0310 TipOfTree TipOfTreeDyn V8: crypto 103.2552+-0.4108 ^ 93.5304+-0.3761 ^ definitely 1.1040x faster deltablue 292.5504+-2.2242 ^ 285.5507+-2.4859 ^ definitely 1.0245x faster earley-boyer 112.3912+-0.2282 ! 114.2831+-0.4305 ! definitely 1.0168x slower raytrace 87.6601+-0.5881 86.8542+-0.2573 regexp 132.9250+-0.5788 ^ 130.0159+-0.5122 ^ definitely 1.0224x faster richards 300.6141+-1.8467 ^ 284.7736+-1.3857 ^ definitely 1.0556x faster splay 145.5287+-1.1222 ? 146.5276+-0.8034 ? <arithmetic> 167.8464+-0.4878 ^ 163.0765+-0.5047 ^ definitely 1.0292x faster <geometric> 150.2703+-0.3533 ^ 146.3499+-0.2865 ^ definitely 1.0268x faster <harmonic> 136.8999+-0.2936 ^ 133.3658+-0.1962 ^ definitely 1.0265x faster TipOfTree TipOfTreeDyn Kraken: ai-astar 1667.9173+-16.6919 ! 1710.0280+-18.8938 ! definitely 1.0252x slower audio-beat-detection 541.8131+-1.8633 ^ 533.0010+-1.7733 ^ definitely 1.0165x faster audio-dft 452.0701+-8.0429 ? 458.7987+-6.4440 ? might be 1.0149x slower audio-fft 421.3918+-0.8967 ^ 416.0566+-0.8896 ^ definitely 1.0128x faster audio-oscillator 403.1402+-0.6423 ^ 372.7753+-4.5984 ^ definitely 1.0815x faster imaging-darkroom 585.7165+-13.8667 ^ 467.8477+-1.8063 ^ definitely 1.2519x faster imaging-desaturate 652.0937+-0.1374 ^ 251.4218+-0.0225 ^ definitely 2.5936x faster imaging-gaussian-blur 1844.7490+-1.9134 1844.4333+-1.5254 json-parse-financial 63.1134+-0.3853 62.6635+-0.2933 json-stringify-tinderbox 83.7694+-0.3490 83.7252+-0.4869 stanford-crypto-aes 166.9100+-1.2094 ^ 164.1889+-0.4856 ^ definitely 1.0166x faster stanford-crypto-ccm 131.0104+-0.6111 ? 131.2150+-1.6770 ? stanford-crypto-pbkdf2 370.9821+-1.3787 ! 436.4344+-3.0435 ! definitely 1.1764x slower stanford-crypto-sha256-iterative 143.6550+-0.1511 ! 163.3373+-0.4790 ! definitely 1.1370x slower <arithmetic> 537.7380+-0.8751 ^ 506.8519+-1.8089 ^ definitely 1.0609x faster <geometric> 342.1178+-0.8278 ^ 319.0591+-0.9121 ^ definitely 1.0723x faster <harmonic> 217.5167+-0.4851 ^ 210.7909+-0.6950 ^ definitely 1.0319x faster TipOfTree TipOfTreeDyn All benchmarks: <arithmetic> 189.2096+-0.2348 ^ 179.2756+-0.5298 ^ definitely 1.0554x faster <geometric> 32.4465+-0.0987 ^ 31.5542+-0.0959 ^ definitely 1.0283x faster <harmonic> 8.7337+-0.0514 8.6626+-0.0538 Tests in browser: FROM = TipOfTree TO = Tiered Compilation TEST COMPARISON FROM TO DETAILS ============================================================================= ** TOTAL **: ?? 191.3ms +/- 1.0% 191.6ms +/- 0.6% not conclusive: might be *1.002x as slow* ============================================================================= 3d: - 25.1ms +/- 0.9% 25.0ms +/- 0.0% cube: - 9.0ms +/- 0.0% 9.0ms +/- 0.0% morph: - 8.0ms +/- 0.0% 8.0ms +/- 0.0% raytrace: - 8.1ms +/- 2.8% 8.0ms +/- 0.0% access: ?? 22.8ms +/- 3.2% 23.5ms +/- 2.6% not conclusive: might be *1.031x as slow* binary-trees: *1.21x as slow* 2.4ms +/- 15.4% 2.9ms +/- 7.8% significant fannkuch: - 12.8ms +/- 2.4% 12.8ms +/- 2.4% nbody: ?? 4.6ms +/- 8.0% 4.7ms +/- 7.3% not conclusive: might be *1.022x as slow* nsieve: ?? 3.0ms +/- 0.0% 3.1ms +/- 7.3% not conclusive: might be *1.033x as slow* bitops: ?? 15.5ms +/- 5.9% 16.0ms +/- 2.1% not conclusive: might be *1.032x as slow* 3bit-bits-in-byte: - 1.6ms +/- 23.1% 1.5ms +/- 25.1% bits-in-byte: ?? 4.4ms +/- 8.4% 4.5ms +/- 8.4% not conclusive: might be *1.023x as slow* bitwise-and: - 4.0ms +/- 0.0% 4.0ms +/- 0.0% nsieve-bits: *1.091x as slow* 5.5ms +/- 6.8% 6.0ms +/- 0.0% significant controlflow: ?? 2.0ms +/- 0.0% 2.1ms +/- 10.8% not conclusive: might be *1.050x as slow* recursive: ?? 2.0ms +/- 0.0% 2.1ms +/- 10.8% not conclusive: might be *1.050x as slow* crypto: *1.092x as slow* 12.0ms +/- 4.0% 13.1ms +/- 3.1% significant aes: *1.125x as slow* 7.2ms +/- 4.2% 8.1ms +/- 2.8% significant md5: ?? 2.6ms +/- 14.2% 2.9ms +/- 7.8% not conclusive: might be *1.115x as slow* sha1: - 2.2ms +/- 13.7% 2.1ms +/- 10.8% date: - 23.0ms +/- 2.1% 22.7ms +/- 1.5% format-tofte: ?? 12.8ms +/- 3.5% 12.9ms +/- 3.1% not conclusive: might be *1.008x as slow* format-xparb: 1.041x as fast 10.2ms +/- 3.0% 9.8ms +/- 3.1% significant math: - 20.5ms +/- 3.8% 20.2ms +/- 2.2% cordic: - 7.2ms +/- 6.3% 7.1ms +/- 3.2% partial-sums: - 10.6ms +/- 3.5% 10.3ms +/- 3.4% spectral-norm: ?? 2.7ms +/- 12.8% 2.8ms +/- 10.8% not conclusive: might be *1.037x as slow* regexp: - 11.9ms +/- 3.4% 11.4ms +/- 3.2% dna: - 11.9ms +/- 3.4% 11.4ms +/- 3.2% string: - 58.5ms +/- 1.9% 57.6ms +/- 2.0% base64: 1.167x as fast 7.7ms +/- 8.8% 6.6ms +/- 5.6% significant fasta: *1.059x as slow* 8.5ms +/- 4.4% 9.0ms +/- 0.0% significant tagcloud: ?? 12.6ms +/- 2.9% 12.9ms +/- 1.8% not conclusive: might be *1.024x as slow* unpack-code: ?? 21.3ms +/- 2.3% 21.4ms +/- 3.2% not conclusive: might be *1.005x as slow* validate-input: 1.091x as fast 8.4ms +/- 5.9% 7.7ms +/- 4.5% significant TipOfTree: Score: 4660 Richards: 4084 DeltaBlue: 3587 Crypto: 9954 RayTrace: 5228 EarleyBoyer: 8052 RegExp: 1787 Splay: 4352 TieredCompilation: (3% faster) Score: 4806 Richards: 4377 DeltaBlue: 3623 Crypto: 10748 RayTrace: 5416 EarleyBoyer: 7892 RegExp: 1785 Splay: 4555 FROM = TipOfTree TO = Tiered Compilation TEST COMPARISON FROM TO DETAILS ==================================================================================== ** TOTAL **: 1.069x as fast 7619.7ms +/- 0.2% 7129.9ms +/- 0.4% significant ==================================================================================== ai: ?? 1690.9ms +/- 1.1% 1717.7ms +/- 1.5% might be *1.016x as slow* astar: ?? 1690.9ms +/- 1.1% 1717.7ms +/- 1.5% might be *1.016x as slow* audio: 1.033x as fast 1842.6ms +/- 0.2% 1784.4ms +/- 0.3% significant beat-detection: 1.024x as fast 563.0ms +/- 0.5% 549.6ms +/- 0.7% significant dft: *1.009x as slow* 450.9ms +/- 0.3% 455.1ms +/- 0.7% significant fft: 1.016x as fast 425.6ms +/- 0.7% 419.0ms +/- 0.6% significant oscillator: 1.118x as fast 403.1ms +/- 0.4% 360.7ms +/- 0.4% significant imaging: 1.21x as fast 3095.3ms +/- 0.1% 2557.2ms +/- 0.2% significant gaussian-blur: 1.002x as fast 1819.6ms +/- 0.1% 1816.3ms +/- 0.1% significant darkroom: 1.27x as fast 622.6ms +/- 0.3% 488.9ms +/- 1.2% significant desaturate: 2.59x as fast 653.1ms +/- 0.2% 252.0ms +/- 0.4% significant json: - 159.1ms +/- 0.6% 159.1ms +/- 0.3% parse-financial: - 75.2ms +/- 1.1% 74.6ms +/- 0.5% stringify-tinderbox: *1.007x as slow* 83.9ms +/- 0.5% 84.5ms +/- 0.4% significant stanford: *1.096x as slow* 831.8ms +/- 0.5% 911.5ms +/- 0.3% significant crypto-aes: ?? 166.2ms +/- 1.3% 167.4ms +/- 1.5% might be *1.007x as slow* crypto-ccm: - 142.5ms +/- 1.1% 142.0ms +/- 0.8% crypto-pbkdf2: *1.167x as slow* 373.3ms +/- 0.8% 435.8ms +/- 0.7% significant crypto-sha256-iterative: *1.110x as slow* 149.8ms +/- 0.7% 166.3ms +/- 0.7% significant

Attachments
the patch (1.27 KB, patch) 2011-09-14 18:18 PDT, Filip Pizlo	webkit-ews: commit-queue-	Details Formatted Diff Diff
the patch - fix everything (1.34 KB, patch) 2011-09-14 18:39 PDT, Filip Pizlo	webkit-ews: commit-queue-	Details Formatted Diff Diff
the patch - for real this time (2.09 KB, patch) 2011-09-14 19:05 PDT, Filip Pizlo	sam: review+	Details Formatted Diff Diff
Show Obsolete (2) View All Add attachment proposed patch, testcase, etc.

Filip Pizlo

Comment 1 2011-09-14 18:18:55 PDT

Created attachment 107435 [details] the patch

Early Warning System Bot

Comment 2 2011-09-14 18:36:43 PDT

Comment on attachment 107435 [details] the patch Attachment 107435 [details] did not pass qt-ews (qt): Output: http://queues.webkit.org/results/9661764

Filip Pizlo

Comment 3 2011-09-14 18:39:04 PDT

Created attachment 107437 [details] the patch - fix everything

Early Warning System Bot

Comment 4 2011-09-14 18:51:18 PDT

Comment on attachment 107437 [details] the patch - fix everything Attachment 107437 [details] did not pass qt-ews (qt): Output: http://queues.webkit.org/results/9655784

Filip Pizlo

Comment 5 2011-09-14 19:05:04 PDT

Created attachment 107442 [details] the patch - for real this time

Filip Pizlo

Comment 6 2011-09-14 20:42:14 PDT

Will wait for the bots to go green before committing, since I'm not yet 100% sure that I've got all those #ifdef's right.

Filip Pizlo

Comment 7 2011-09-15 11:54:38 PDT

Landed in r95206. The last numbers (comparing 95205 to 95206): Benchmark report for SunSpider, V8, and Kraken. VMs tested: "TipOfTree" at /Volumes/Data/pizlo/octonary/OpenSource/WebKitBuild/Release/jsc "TieredCompilation" at /Volumes/Data/pizlo/quinary/OpenSource/WebKitBuild/Release/jsc Collected 12 samples per benchmark/VM, with 4 VM invocations per benchmark. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in milliseconds. TipOfTree TieredCompilation SunSpider: 3d-cube 7.9594+-0.2150 7.9262+-0.1658 3d-morph 7.5548+-0.1317 ? 7.5557+-0.1786 ? 3d-raytrace 8.2024+-0.3451 7.6801+-0.2302 might be 1.0680x faster access-binary-trees 2.2097+-0.0229 ! 2.2953+-0.0507 ! definitely 1.0387x slower access-fannkuch 11.6981+-0.2218 ? 11.8111+-0.2361 ? access-nbody 4.3410+-0.1316 4.2497+-0.1391 might be 1.0215x faster access-nsieve 2.5923+-0.0997 ? 2.6411+-0.0757 ? might be 1.0188x slower bitops-3bit-bits-in-byte 1.7042+-0.0569 ? 1.7102+-0.0685 ? bitops-bits-in-byte 2.7652+-0.0643 ? 2.8248+-0.1128 ? might be 1.0215x slower bitops-bitwise-and 3.7014+-0.1329 3.6661+-0.1160 bitops-nsieve-bits 5.4193+-0.0787 5.2836+-0.1220 might be 1.0257x faster controlflow-recursive 1.9813+-0.0341 ? 2.0904+-0.0850 ? might be 1.0551x slower crypto-aes 6.7605+-0.2467 ? 7.1447+-0.3279 ? might be 1.0568x slower crypto-md5 2.7752+-0.0483 ? 3.3646+-0.6840 ? might be 1.2124x slower crypto-sha1 2.2489+-0.0621 2.1552+-0.0704 might be 1.0435x faster date-format-tofte 11.2852+-0.3376 ^ 10.3442+-0.2719 ^ definitely 1.0910x faster date-format-xparb 8.8027+-0.2859 8.7357+-0.2227 math-cordic 6.3806+-0.1088 6.2527+-0.1303 might be 1.0204x faster math-partial-sums 7.6559+-0.1588 7.3674+-0.1612 might be 1.0392x faster math-spectral-norm 2.4454+-0.0401 ! 2.5832+-0.0443 ! definitely 1.0563x slower regexp-dna 11.1329+-0.2861 10.7977+-0.2008 might be 1.0310x faster string-base64 5.8202+-0.1199 5.8175+-0.1956 string-fasta 7.1207+-0.0827 ! 8.5540+-0.2553 ! definitely 1.2013x slower string-tagcloud 11.8230+-0.2290 ? 11.8602+-0.2660 ? string-unpack-code 19.1518+-0.4605 19.1024+-0.4931 string-validate-input 7.5095+-0.3377 ^ 6.6418+-0.1659 ^ definitely 1.1307x faster <arithmetic> 6.5785+-0.0437 6.5560+-0.0377 <geometric> 5.3869+-0.0303 ? 5.4105+-0.0375 ? <harmonic> 4.3522+-0.0249 ? 4.4074+-0.0392 ? might be 1.0127x slower TipOfTree TieredCompilation V8: crypto 90.8545+-0.8925 ^ 85.2666+-2.3793 ^ definitely 1.0655x faster deltablue 265.8815+-3.5027 ^ 247.8677+-2.3100 ^ definitely 1.0727x faster earley-boyer 94.4787+-0.7301 ? 95.5216+-0.8552 ? might be 1.0110x slower raytrace 76.1163+-1.1740 ^ 71.9430+-0.4490 ^ definitely 1.0580x faster regexp 108.4889+-1.3826 107.0564+-0.5068 might be 1.0134x faster richards 239.9382+-2.8300 ^ 220.9082+-2.7744 ^ definitely 1.0861x faster splay 99.3576+-0.2353 ? 99.8593+-0.4493 ? <arithmetic> 139.3022+-0.6465 ^ 132.6318+-0.5601 ^ definitely 1.0503x faster <geometric> 123.9271+-0.5100 ^ 119.2335+-0.4920 ^ definitely 1.0394x faster <harmonic> 112.9873+-0.5110 ^ 109.3138+-0.5171 ^ definitely 1.0336x faster TipOfTree TieredCompilation Kraken: ai-astar 1220.2982+-33.6219 ^ 651.2664+-13.5760 ^ definitely 1.8737x faster audio-beat-detection 481.6183+-3.7772 476.8139+-5.5267 might be 1.0101x faster audio-dft 423.7553+-5.9952 417.3201+-3.1266 might be 1.0154x faster audio-fft 369.7725+-0.7480 369.4925+-4.5536 audio-oscillator 392.0498+-16.2092 ^ 323.1123+-7.5327 ^ definitely 1.2134x faster imaging-darkroom 538.4024+-9.1255 ^ 417.7482+-3.7546 ^ definitely 1.2888x faster imaging-desaturate 629.2546+-14.4686 ^ 211.4487+-1.9027 ^ definitely 2.9759x faster imaging-gaussian-blur 1729.7578+-6.4878 ? 1739.8427+-19.4150 ? json-parse-financial 48.4246+-0.4196 ? 49.0291+-0.8502 ? might be 1.0125x slower json-stringify-tinderbox 70.2918+-2.9485 68.1125+-0.4787 might be 1.0320x faster stanford-crypto-aes 147.8755+-2.6111 ^ 144.7200+-0.4577 ^ definitely 1.0218x faster stanford-crypto-ccm 113.2348+-1.2837 111.8843+-0.7783 might be 1.0121x faster stanford-crypto-pbkdf2 339.8100+-9.8956 ! 404.5472+-2.2667 ! definitely 1.1905x slower stanford-crypto-sha256-iterative 132.5923+-0.9317 ! 150.5893+-1.3005 ! definitely 1.1357x slower <arithmetic> 474.0813+-3.5488 ^ 395.4234+-1.9613 ^ definitely 1.1989x faster <geometric> 302.5975+-1.5819 ^ 263.5863+-0.6790 ^ definitely 1.1480x faster <harmonic> 185.2691+-1.1003 ^ 175.9015+-0.5961 ^ definitely 1.0533x faster TipOfTree TieredCompilation All benchmarks: <arithmetic> 165.6020+-1.0020 ^ 141.1661+-0.5955 ^ definitely 1.1731x faster <geometric> 28.5295+-0.1039 ^ 27.2891+-0.1128 ^ definitely 1.0455x faster <harmonic> 7.6904+-0.0429 ? 7.7777+-0.0675 ? might be 1.0114x slower

Note You need to log in before you can comment on or make changes to this bug.