Bug 149301

Summary: add a regress test for richards with try/catch.
Product: WebKit Reporter: Saam Barati <sbarati>
Component: JavaScriptCoreAssignee: Saam Barati <sbarati>
Status: RESOLVED FIXED    
Severity: Normal CC: benjamin, commit-queue, fpizlo, ggaren, keith_miller, mark.lam, msaboff, oliver, ysuzuki
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 149329    
Bug Blocks:    
Attachments:
Description Flags
patch none

Description Saam Barati 2015-09-17 15:08:02 PDT
Forgot to add this in DFG try/catch patch.
Comment 1 Saam Barati 2015-09-17 16:45:59 PDT
Created attachment 261455 [details]
patch
Comment 2 Oliver Hunt 2015-09-17 16:50:31 PDT
Comment on attachment 261455 [details]
patch

How many exceptions do you actually hit?
Comment 3 Saam Barati 2015-09-17 17:22:28 PDT
(In reply to comment #2)
> Comment on attachment 261455 [details]
> patch
> 
> How many exceptions do you actually hit?

3486.

I've played with the frequency number a bit.
Obviously, as that number gets smaller, the
tests get slower. And as it becomes larger,
and exceptions are less frequent, the test
becomes faster.
Comment 4 Saam Barati 2015-09-17 17:44:31 PDT
Here are some perf results with different exception frequencies. 
The only affects richards-try-catch benchmark.
Also, r189935 is the try/catch patch. And r189934 is the revision right
before this patch.


------------------


With the frequency being 1/10000
VMs tested:
"og" at /Volumes/Data/WK/c/OpenSource/WebKitBuild/Release/jsc (r189934)
"tryCatch" at /Volumes/Data/WK/b/OpenSource/WebKitBuild/Release/jsc (r189935)

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                      og                     tryCatch                                     

richards-empty-try-catch      194.9911+-3.4160     ^     76.2141+-3.4663        ^ definitely 2.5585x faster
richards-try-catch            294.3329+-9.1718     ^    211.1737+-5.1058        ^ definitely 1.3938x faster

<geometric>                   239.5515+-3.4016     ^    126.8551+-3.7277        ^ definitely 1.8884x faster


--------------------


With the frequency being 1/5000 (what's in the actual patch)
VMs tested:
"og" at /Volumes/Data/WK/c/OpenSource/WebKitBuild/Release/jsc (r189934)
"tryCatch" at /Volumes/Data/WK/b/OpenSource/WebKitBuild/Release/jsc (r189935)

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                      og                     tryCatch                                     

richards-empty-try-catch      195.6956+-4.6587     ^     76.0357+-2.4998        ^ definitely 2.5737x faster
richards-try-catch            309.5485+-6.6035     ^    235.3614+-16.2632       ^ definitely 1.3152x faster

<geometric>                   246.1172+-4.3429     ^    133.7329+-3.6655        ^ definitely 1.8404x faster


----------------


With the frequency being 1/2500
VMs tested:
"og" at /Volumes/Data/WK/c/OpenSource/WebKitBuild/Release/jsc (r189934)
"tryCatch" at /Volumes/Data/WK/b/OpenSource/WebKitBuild/Release/jsc (r189935)

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                      og                     tryCatch                                     

richards-empty-try-catch      194.6537+-3.4410     ^     76.7055+-5.4004        ^ definitely 2.5377x faster
richards-try-catch            330.1699+-3.1142     ^    261.1393+-5.1116        ^ definitely 1.2643x faster

<geometric>                   253.5082+-2.2087     ^    141.5020+-5.0380        ^ definitely 1.7916x faster


----------------


With the frequency being 1/1000
VMs tested:
"og" at /Volumes/Data/WK/c/OpenSource/WebKitBuild/Release/jsc (r189934)
"tryCatch" at /Volumes/Data/WK/b/OpenSource/WebKitBuild/Release/jsc (r189935)

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                      og                     tryCatch                                     

richards-empty-try-catch      198.9267+-10.7999    ^     77.1111+-3.5368        ^ definitely 2.5797x faster
richards-try-catch            422.8513+-5.3213     ^    349.4740+-8.3545        ^ definitely 1.2100x faster

<geometric>                   289.9953+-8.0337     ^    164.1356+-3.0591        ^ definitely 1.7668x faster


---------------


With the frequency being 1/500
VMs tested:
"og" at /Volumes/Data/WK/c/OpenSource/WebKitBuild/Release/jsc (r189934)
"tryCatch" at /Volumes/Data/WK/b/OpenSource/WebKitBuild/Release/jsc (r189935)

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                      og                     tryCatch                                     

richards-empty-try-catch      195.8364+-7.6551     ^     76.0762+-3.0992        ^ definitely 2.5742x faster
richards-try-catch            524.2897+-10.7847    ^    467.4330+-11.8313       ^ definitely 1.1216x faster

<geometric>                   320.4008+-6.0738     ^    188.5623+-4.9270        ^ definitely 1.6992x faster


------------------


With the frequency being 1/100
VMs tested:
"og" at /Volumes/Data/WK/c/OpenSource/WebKitBuild/Release/jsc (r189934)
"tryCatch" at /Volumes/Data/WK/b/OpenSource/WebKitBuild/Release/jsc (r189935)

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                      og                     tryCatch                                     

richards-empty-try-catch      196.4677+-11.6653    ^     75.4759+-0.5515        ^ definitely 2.6031x faster
richards-try-catch           1538.8857+-21.0152    ^   1493.2333+-15.2021       ^ definitely 1.0306x faster

<geometric>                   549.7620+-14.2294    ^    335.7113+-2.2949        ^ definitely 1.6376x faster
Comment 6 WebKit Commit Bot 2015-09-17 18:35:14 PDT
Comment on attachment 261455 [details]
patch

Clearing flags on attachment: 261455

Committed r189956: <http://trac.webkit.org/changeset/189956>
Comment 7 WebKit Commit Bot 2015-09-17 18:35:18 PDT
All reviewed patches have been landed.  Closing bug.
Comment 8 Alexey Proskuryakov 2015-09-17 19:49:23 PDT
> Forgot to add this in DFG try/catch patch.

Not sure if this actually depends on that patch, but given this comment, I'll roll this out too.
Comment 9 WebKit Commit Bot 2015-09-17 19:52:37 PDT
Re-opened since this is blocked by bug 149329
Comment 10 Saam Barati 2015-09-18 16:44:04 PDT
landed in:
http://trac.webkit.org/changeset/189996