Bug 111797 - REGRESSION (r143759): 40% JSBench regression, 20% Octane/closure regression, 40% Octane/jquery regression, 2% Octane regression
Summary: REGRESSION (r143759): 40% JSBench regression, 20% Octane/closure regression, ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Geoffrey Garen
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-07 16:58 PST by Geoffrey Garen
Modified: 2013-03-07 19:03 PST (History)
3 users (show)

See Also:


Attachments
Patch (4.50 KB, patch)
2013-03-07 17:04 PST, Geoffrey Garen
oliver: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Geoffrey Garen 2013-03-07 16:58:29 PST
REGRESSION (r143759): 40% JSBench regression, 20% Octane/closure regression, 40% Octane/jquery regression, 2% Octane regression
Comment 1 Geoffrey Garen 2013-03-07 16:58:57 PST
Seen on the bot, which runs:

        run-jsc-benchmarks --outer 2 --inner 2
Comment 2 Geoffrey Garen 2013-03-07 16:59:47 PST
In this configuration, the cache's starting guess of 1MB dominates program execution, before any of the cache's smarts kick in.
Comment 3 Geoffrey Garen 2013-03-07 17:04:59 PST
Created attachment 192115 [details]
Patch
Comment 4 Geoffrey Garen 2013-03-07 17:06:13 PST
Tested vs r143758, r143759+this patch is no performance regression:

Benchmark report for SunSpider, V8Spider, Octane, Kraken, JSBench, JSRegress, and DSP on garen (MacPro5,1).

VMs tested:
"BASE" at /Volumes/Big/ggaren/baseline-webkit/WebKitBuild/Release/DumpRenderTree (r143758)
"PATCH" at /Volumes/Big/ggaren/webkit/WebKitBuild/Release/DumpRenderTree (r143949)

Collected 4 samples per benchmark/VM, with 2 VM invocations per benchmark. Emitted a call to gc() between sample
measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific preciseTime() function to get
microsecond-level timing. Reporting benchmark execution times with 95% confidence intervals in milliseconds.

                                                        BASE                     PATCH                                       
SunSpider:
   3d-cube                                        10.8445+-0.7846           10.8033+-0.9457        
   3d-morph                                        9.2441+-0.1685     ?      9.3491+-0.1444        ? might be 1.0113x slower
   3d-raytrace                                    11.7272+-0.8772           11.6065+-1.0136          might be 1.0104x faster
   access-binary-trees                             2.4954+-0.9608     ?      2.5096+-0.9218        ?
   access-fannkuch                                 7.5344+-0.0534     ?      7.5380+-0.0824        ?
   access-nbody                                    4.7698+-0.1879            4.7188+-0.1338          might be 1.0108x faster
   access-nsieve                                   4.8518+-0.0417            4.8140+-0.1543        
   bitops-3bit-bits-in-byte                        1.7240+-0.0130     ?      1.7407+-0.0738        ?
   bitops-bits-in-byte                             6.4092+-0.0801     ?      6.4232+-0.0434        ?
   bitops-bitwise-and                              2.2124+-0.0088     ?      2.3375+-0.3203        ? might be 1.0565x slower
   bitops-nsieve-bits                              4.0410+-0.0372            4.0195+-0.0206        
   controlflow-recursive                           2.8617+-0.0298     ?      2.9075+-0.0377        ? might be 1.0160x slower
   crypto-aes                                      8.7312+-1.1281     ?      8.8525+-0.8099        ? might be 1.0139x slower
   crypto-md5                                      4.1807+-0.3279     ?      4.1904+-0.0810        ?
   crypto-sha1                                     3.5004+-0.0847     ?      3.5198+-0.1011        ?
   date-format-tofte                              17.6857+-3.1110     ?     17.7252+-2.9684        ?
   date-format-xparb                              16.7502+-2.1209     ?     16.8943+-2.2214        ?
   math-cordic                                     3.7443+-0.0577            3.7332+-0.0387        
   math-partial-sums                              11.9683+-0.0915     ?     12.0475+-0.0688        ?
   math-spectral-norm                              3.1191+-0.0279     ?      3.1193+-0.0128        ?
   regexp-dna                                     10.7065+-1.4032     ?     10.7498+-1.6171        ?
   string-base64                                   6.5704+-1.8249            6.4070+-1.5885          might be 1.0255x faster
   string-fasta                                   11.1532+-0.5612           11.0920+-0.5646        
   string-tagcloud                                15.0109+-0.4949     ?     15.0544+-0.3297        ?
   string-unpack-code                             28.1624+-1.2837     ?     28.3555+-1.1793        ?
   string-validate-input                           9.9772+-0.3993     ?     10.0818+-0.4584        ? might be 1.0105x slower

   <arithmetic> *                                  8.4606+-0.3793     ?      8.4842+-0.4918        ? might be 1.0028x slower
   <geometric>                                     6.6440+-0.1988     ?      6.6702+-0.3672        ? might be 1.0039x slower
   <harmonic>                                      5.1917+-0.1548     ?      5.2317+-0.3022        ? might be 1.0077x slower

                                                        BASE                     PATCH                                       
V8Spider:
   crypto                                         83.2711+-0.1185     ?     83.6557+-1.1847        ?
   deltablue                                     130.6743+-1.1040          130.4390+-0.8337        
   earley-boyer                                   84.7060+-1.5650     ?     84.9540+-2.0482        ?
   raytrace                                       64.6742+-0.7971     ?     64.8422+-0.7115        ?
   regexp                                         99.7057+-0.6723           99.4838+-1.8482        
   richards                                      113.2200+-1.4973          112.7955+-0.5902        
   splay                                          68.0616+-8.2055     ?     69.2809+-8.0491        ? might be 1.0179x slower

   <arithmetic>                                   92.0447+-1.5513     ?     92.2073+-1.6689        ? might be 1.0018x slower
   <geometric> *                                  89.4217+-1.9567     ?     89.6802+-2.0215        ? might be 1.0029x slower
   <harmonic>                                     86.9216+-2.3685     ?     87.2713+-2.3732        ? might be 1.0040x slower

                                                        BASE                     PATCH                                       
Octane and V8v7:
   encrypt                                        0.44408+-0.00092    ?     0.44435+-0.00098       ?
   decrypt                                        7.93859+-0.08746          7.89670+-0.05854       
   deltablue                             x2       0.59119+-0.02061    ?     0.59236+-0.01795       ?
   earley                                         0.86965+-0.02265          0.86194+-0.01804       
   boyer                                         12.55236+-0.13026         12.47342+-0.07120       
   raytrace                              x2       4.79423+-1.06122    ?     4.88771+-1.14686       ? might be 1.0195x slower
   regexp                                x2      30.25632+-0.53173    ?    30.38146+-1.05853       ?
   richards                              x2       0.28592+-0.00116          0.28522+-0.00079       
   splay                                 x2       0.69100+-0.05211          0.68795+-0.01002       
   navier-stokes                         x2      11.54579+-0.07162         11.53851+-0.09948       
   closure                                        0.36877+-0.01537          0.31873+-0.08159         might be 1.1570x faster
   jquery                                         4.90589+-0.84145          4.75864+-1.10108         might be 1.0309x faster
   gbemu                                 x2     257.86023+-7.74264        255.38346+-7.76150       
   mandreel                              x2     173.86414+-0.98129        173.86199+-0.65114       
   pdfjs                                 x2     126.44341+-0.77217    ?   126.47745+-0.40573       ?
   box2d                                 x2      33.79110+-0.50082    ?    33.93483+-0.48910       ?

V8v7:
   <arithmetic>                                   7.38335+-0.20504    ?     7.40143+-0.17180       ? might be 1.0024x slower
   <geometric> *                                  2.42741+-0.07452    ?     2.42974+-0.06818       ? might be 1.0010x slower
   <harmonic>                                     0.91187+-0.01510          0.91072+-0.00498         might be 1.0013x faster

Octane including V8v7:
   <arithmetic>                                  50.28177+-0.51132         50.10829+-0.52922         might be 1.0035x faster
   <geometric> *                                  7.65436+-0.12320          7.60030+-0.10402         might be 1.0071x faster
   <harmonic>                                     1.26463+-0.01875          1.23414+-0.04579         might be 1.0247x faster

                                                        BASE                     PATCH                                       
Kraken:
   ai-astar                                       502.670+-2.658            501.996+-1.897         
   audio-beat-detection                           238.697+-1.598      ?     244.784+-8.832         ? might be 1.0255x slower
   audio-dft                                      300.669+-0.783            298.920+-1.627         
   audio-fft                                      139.287+-0.227            139.242+-0.060         
   audio-oscillator                               239.892+-0.754            239.858+-1.058         
   imaging-darkroom                               286.448+-1.849      ?     293.626+-5.750         ? might be 1.0251x slower
   imaging-desaturate                             149.738+-0.544            149.587+-0.394         
   imaging-gaussian-blur                          409.483+-0.650      ?     409.964+-0.764         ?
   json-parse-financial                            76.690+-0.298      ?      77.636+-0.723         ? might be 1.0123x slower
   json-stringify-tinderbox                        97.542+-0.335      ^      96.687+-0.272         ^ definitely 1.0088x faster
   stanford-crypto-aes                            121.043+-0.905            119.419+-0.784           might be 1.0136x faster
   stanford-crypto-ccm                            138.379+-0.500      !     139.402+-0.437         ! definitely 1.0074x slower
   stanford-crypto-pbkdf2                         264.232+-2.113            262.253+-2.080         
   stanford-crypto-sha256-iterative               121.310+-0.742      ?     121.636+-1.425         ?

   <arithmetic> *                                 220.434+-0.409      ?     221.072+-0.803         ? might be 1.0029x slower
   <geometric>                                    190.846+-0.279      ?     191.319+-0.633         ? might be 1.0025x slower
   <harmonic>                                     166.129+-0.267      ?     166.446+-0.509         ? might be 1.0019x slower

                                                        BASE                     PATCH                                       
JSBench:
   amazon                                         10.7500+-0.7956     ?     11.0000+-0.0000        ? might be 1.0233x slower
   facebook                                       41.5000+-2.7561           41.5000+-2.7561        
   google                                         89.0000+-1.8374           88.5000+-2.7561        
   twitter                                        10.5000+-0.9187           10.2500+-0.7956          might be 1.0244x faster
   yahoo                                           3.2500+-0.7956     ?      3.5000+-0.9187        ? might be 1.0769x slower

   <arithmetic> *                                 31.0000+-1.1326           30.9500+-1.2013          might be 1.0016x faster
   <geometric>                                    16.8152+-1.0642     ?     17.0363+-0.8991        ? might be 1.0131x slower
   <harmonic>                                      9.3654+-1.2465     ?      9.7523+-1.3733        ? might be 1.0413x slower

                                                        BASE                     PATCH                                       
JSRegress:
   adapt-to-double-divide                         21.3252+-0.0940           21.3044+-0.0361        
   aliased-arguments-getbyval                      1.0646+-0.0209            1.0378+-0.0238          might be 1.0259x faster
   allocate-big-object                             3.5673+-3.5418     ?      4.6005+-3.9606        ? might be 1.2896x slower
   arity-mismatch-inlining                         0.7907+-0.0134     ?      0.7963+-0.0366        ?
   array-access-polymorphic-structure              9.9472+-6.0593     ?      9.9867+-6.0416        ?
   array-with-double-add                           5.2810+-0.3585            5.2633+-0.4279        
   array-with-double-increment                     3.8212+-0.0423            3.7784+-0.0571          might be 1.0113x faster
   array-with-double-mul-add                       7.1237+-0.7616     ?      7.1602+-0.7909        ?
   array-with-double-sum                           7.4321+-0.1943            7.3661+-0.0133        
   array-with-int32-add-sub                        9.3740+-1.0129            9.3699+-0.9902        
   array-with-int32-or-double-sum                  7.5025+-0.0903            7.4628+-0.0113        
   big-int-mul                                     4.6415+-0.0966            4.6177+-0.0556        
   boolean-test                                    4.0421+-0.0470            4.0408+-0.0361        
   cast-int-to-double                             13.5284+-0.0299           13.5047+-0.0150        
   cell-argument                                  13.6459+-0.0447           13.6141+-0.0132        
   cfg-simplify                                    3.5595+-0.0212     ?      3.5778+-0.0759        ?
   cmpeq-obj-to-obj-other                         11.4730+-1.1690     ?     11.9147+-0.2717        ? might be 1.0385x slower
   constant-test                                   8.0853+-0.0931            8.0473+-0.2754        
   direct-arguments-getbyval                       0.9675+-0.0602            0.9573+-0.0338          might be 1.0106x faster
   double-pollution-getbyval                      10.1666+-0.0619           10.1593+-0.0192        
   double-pollution-putbyoffset                    5.7504+-1.8409     ?      5.7628+-1.9206        ?
   external-arguments-getbyval                     2.6129+-0.5319            2.6120+-0.5266        
   external-arguments-putbyval                     3.8735+-0.7932            3.8421+-0.8527        
   Float32Array-matrix-mult                       14.3423+-1.1474     ?     14.6804+-1.4607        ? might be 1.0236x slower
   fold-double-to-int                             20.9918+-0.8626           20.8351+-0.4499        
   function-dot-apply                              2.9362+-0.1133     ?      2.9719+-0.0213        ? might be 1.0122x slower
   function-test                                   4.7047+-0.1645            4.6480+-0.1301          might be 1.0122x faster
   get-by-id-chain-from-try-block                  7.1092+-0.1136            7.0693+-0.0321        
   HashMap-put-get-iterate-keys                   86.5285+-1.0078           84.2521+-1.6288          might be 1.0270x faster
   HashMap-put-get-iterate                        87.5262+-2.5336     ?     87.6030+-2.8559        ?
   HashMap-string-put-get-iterate                 78.8381+-2.0048     ?     78.8828+-2.0635        ?
   indexed-properties-in-objects                   4.1038+-0.0188     ?      4.1235+-0.0663        ?
   inline-arguments-access                         1.3083+-0.0171     ?      1.3184+-0.0350        ?
   inline-arguments-local-escape                  24.2919+-0.0986     ?     24.4677+-0.0854        ?
   inline-get-scoped-var                           6.1459+-0.0062     ?      6.1943+-0.1248        ?
   inlined-put-by-id-transition                   14.2435+-0.2155     ?     14.4020+-0.2386        ? might be 1.0111x slower
   int-or-other-abs-then-get-by-val                8.0425+-0.0306            8.0302+-0.0199        
   int-or-other-abs-zero-then-get-by-val          35.8246+-0.7639           35.3892+-0.3974          might be 1.0123x faster
   int-or-other-add-then-get-by-val               10.0778+-0.0253           10.0604+-0.0092        
   int-or-other-add                               10.2256+-0.2170           10.1357+-0.0193        
   int-or-other-div-then-get-by-val                7.3948+-0.0205     ?      7.3955+-0.0068        ?
   int-or-other-max-then-get-by-val                9.3279+-0.5347     ?      9.4453+-0.5487        ? might be 1.0126x slower
   int-or-other-min-then-get-by-val                7.6627+-0.0234            7.6189+-0.0310        
   int-or-other-mod-then-get-by-val                7.3554+-0.0268            7.3547+-0.0421        
   int-or-other-mul-then-get-by-val                6.6501+-0.0152            6.6296+-0.0130        
   int-or-other-neg-then-get-by-val                7.9337+-0.2913            7.9233+-0.2668        
   int-or-other-neg-zero-then-get-by-val          34.7742+-0.1800     ?     35.0142+-0.4512        ?
   int-or-other-sub-then-get-by-val               10.0842+-0.0407           10.0565+-0.0036        
   int-or-other-sub                                8.1322+-0.0147     ?      8.3122+-0.3412        ? might be 1.0221x slower
   int-overflow-local                             16.1440+-0.6392           15.8088+-0.0405          might be 1.0212x faster
   Int16Array-bubble-sort                         74.5720+-7.3809           72.5685+-0.8320          might be 1.0276x faster
   Int16Array-load-int-mul                         1.7955+-0.0383            1.7850+-0.0086        
   Int8Array-load                                  5.1475+-0.0475            5.1412+-0.0625        
   integer-divide                                 14.0141+-0.0611           14.0137+-0.0662        
   integer-modulo                                  2.1102+-0.0654     ?      2.1296+-0.1021        ?
   make-indexed-storage                            4.7345+-1.9224            4.7283+-1.9563        
   method-on-number                               22.1796+-0.8321     ?     22.5731+-1.6184        ? might be 1.0177x slower
   nested-function-parsing                       375.9600+-19.9780    ?    384.5302+-28.2400       ? might be 1.0228x slower
   new-array-buffer-dead                           3.5644+-0.4021     ?      3.6366+-0.3225        ? might be 1.0203x slower
   new-array-buffer-push                          14.3543+-7.7096     ?     14.5480+-8.0693        ? might be 1.0135x slower
   new-array-dead                                 26.8387+-0.0145     ?     26.9267+-0.2253        ?
   new-array-push                                  9.5856+-0.5040     ?      9.6016+-0.7151        ?
   number-test                                     4.0155+-0.0843            4.0056+-0.0865        
   object-closure-call                             8.3179+-0.4477            8.2891+-0.6222        
   object-test                                     4.5630+-0.0333            4.5261+-0.0813        
   poly-stricteq                                  86.6387+-0.5544     ?     87.7812+-3.4300        ? might be 1.0132x slower
   polymorphic-structure                          19.0568+-0.0242     ?     19.0883+-0.0595        ?
   polyvariant-monomorphic-get-by-id              11.8458+-0.0417     ?     11.8679+-0.1097        ?
   rare-osr-exit-on-local                         19.5688+-0.3128           19.4901+-0.0836        
   register-pressure-from-osr                     30.2239+-0.2969           30.0742+-0.0206        
   simple-activation-demo                         33.4773+-0.4188     ?     33.6417+-1.3324        ?
   slow-array-profile-convergence                  4.5443+-0.7483            4.5164+-0.7040        
   slow-convergence                                3.7814+-0.0202            3.7690+-0.0075        
   sparse-conditional                              1.2935+-0.0098            1.2775+-0.0380          might be 1.0125x faster
   splice-to-remove                               47.7029+-0.2172     ?     47.9695+-0.0679        ?
   string-hash                                     2.5328+-0.0310            2.5207+-0.0092        
   string-repeat-arith                            42.1205+-0.5051           41.6890+-0.3246          might be 1.0104x faster
   string-sub                                     81.1405+-0.9768     ?     83.3143+-6.5093        ? might be 1.0268x slower
   string-test                                     3.9695+-0.0523     ?      3.9813+-0.0882        ?
   structure-hoist-over-transitions                4.1459+-1.8641            4.1345+-1.8902        
   tear-off-arguments-simple                       2.0060+-0.0253            2.0049+-0.0099        
   tear-off-arguments                              3.4210+-0.0305     ?      3.4270+-0.0153        ?
   temporal-structure                             19.9715+-0.4221           19.8500+-0.0515        
   to-int32-boolean                               26.9224+-0.1199     ?     26.9274+-0.0376        ?
   undefined-test                                  4.1865+-0.0296     ?      4.1982+-0.0528        ?

   <arithmetic>                                   20.1480+-0.3979     ?     20.2580+-0.6098        ? might be 1.0055x slower
   <geometric> *                                   9.0410+-0.2951     ?      9.0697+-0.3690        ? might be 1.0032x slower
   <harmonic>                                      5.1275+-0.1346     ?      5.1383+-0.1480        ? might be 1.0021x slower

                                                        BASE                     PATCH                                       
DSP:
   filtrr-posterize-tint                          50.9678+-3.7282           50.8928+-3.4149        
   filtrr-tint-contrast-sat-bright                71.6526+-1.8568     ?     71.8545+-1.0988        ?
   filtrr-tint-sat-adj-contr-mult                 86.7718+-7.5529           85.7352+-6.0466          might be 1.0121x faster
   filtrr-blur-overlay-sat-contr                 227.6051+-4.7696     ?    230.4676+-11.7565       ? might be 1.0126x slower
   filtrr-sat-blur-mult-sharpen-contr            259.4548+-17.7710    ?    261.1887+-15.3500       ?
   filtrr-sepia-bias                              34.7023+-1.9761           34.5577+-2.0879        
   route9-vp8                            x5     1198.1892+-14.6222        1186.0369+-12.7893         might be 1.0102x faster
   starfield                             x5     1166.9689+-10.9665        1161.9007+-14.5436       
   zynaps-quake3                         x5     1364.4535+-50.7654        1319.2670+-105.8500        might be 1.0343x faster
   zynaps-mandelbrot                     x5     1144.3880+-12.4426        1140.5120+-17.3210       

   <arithmetic>                                  965.4289+-10.1073         952.8184+-27.3303         might be 1.0132x faster
   <geometric> *                                 671.6801+-5.3838          665.0883+-11.6797         might be 1.0099x faster
   <harmonic>                                    263.4108+-10.5501         262.4038+-8.9971          might be 1.0038x faster

                                                        BASE                     PATCH                                       
All benchmarks:
   <arithmetic>                                  170.5102+-1.3270          168.8562+-3.6162          might be 1.0098x faster
   <geometric>                                    21.2481+-0.4398     ?     21.2524+-0.4938        ? might be 1.0002x slower
   <harmonic>                                      4.4008+-0.0773            4.3572+-0.0079          might be 1.0100x faster

                                                        BASE                     PATCH                                       
Geomean of preferred means:
   <scaled-result>                                42.2464+-0.5918           42.2031+-0.5879          might be 1.0010x faster
Comment 5 Geoffrey Garen 2013-03-07 19:03:22 PST
Committed r145171: <http://trac.webkit.org/changeset/145171>