RESOLVED WONTFIX 78216
[Chromium] REGRESSION(r107164-r107171): 12% regression on Dromaeo core eval test 
https://bugs.webkit.org/show_bug.cgi?id=78216
Summary [Chromium] REGRESSION(r107164-r107171): 12% regression on Dromaeo core eval test
Attachments
Add attachment proposed patch, testcase, etc.
Ryosuke Niwa
Comment 1 2012-02-09 02:11:38 PST
Note the bot had some issues until yesterday afternoon so the results prior to that is kind of bogus and has an insanely large variance.
Ryosuke Niwa
Comment 2 2012-02-09 02:17:53 PST
+gardeners for a potential perf regression. We don't seem to run core eval test on Chromium perf bots but it may show up somewhere.
Adam Klein
Comment 3 2012-02-09 09:32:45 PST
The only perf-related change in r107170 would be an occasional call to the virtual ScriptExecutionContext::isDocument() method. But I really doubt this would show up anywhere...do you have particular culprit(s) in the Chromium roll?
Ryosuke Niwa
Comment 4 2012-02-09 19:35:19 PST
r107142 Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3886.39443236 ms median= 0.0 ms, stdev= 61.7359952295 ms, min= 3781.77567298 ms, max= 3943.24752475 ms wdiff not found. Install using MacPorts or some other means Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3920.19970262 ms median= 0.0 ms, stdev= 19.6743600988 ms, min= 3904.85429142 ms, max= 3953.55864811 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3976.55652424 ms median= 0.0 ms, stdev= 39.5644971639 ms, min= 3905.77268195 ms, max= 3999.31318137 ms r107160 Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3862.62232759 ms median= 0.0 ms, stdev= 22.3278233281 ms, min= 3824.0 ms, max= 3882.84630739 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3967.68778445 ms median= 0.0 ms, stdev= 33.9811638656 ms, min= 3914.71713147 ms, max= 3993.07601185 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3875.49859065 ms median= 0.0 ms, stdev= 8.62363597302 ms, min= 3862.62189055 ms, max= 3886.16221563 ms r107166 Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3928.25889514 ms median= 0.0 ms, stdev= 25.440920616 ms, min= 3907.70517928 ms, max= 3971.18397626 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3935.19959675 ms median= 0.0 ms, stdev= 27.6726034674 ms, min= 3914.5526839 ms, max= 3984.24752475 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 3976.60576972 ms median= 0.0 ms, stdev= 37.9178850146 ms, min= 3929.47169811 ms, max= 4006.08695652 ms r107170 Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 4155.18976733 ms median= 0.0 ms, stdev= 27.084515622 ms, min= 4112.7666999 ms, max= 4182.45084409 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 4169.07803417 ms median= 0.0 ms, stdev= 24.2431246092 ms, min= 4139.45084409 ms, max= 4207.6812749 ms Running Dromaeo/dromaeo-core-eval.html (1 of 1) RESULT Dromaeo: dromaeo-core-eval= 4145.07851511 ms median= 0.0 ms, stdev= 28.5225952808 ms, min= 4100.07509881 ms, max= 4167.2864222 ms
Ryosuke Niwa
Comment 5 2012-02-09 19:36:12 PST
At this point, it's pretty clear to me that this regression is due to r107170.
Ryosuke Niwa
Comment 6 2012-02-09 19:42:29 PST
Wait, this isn't right. The score should go down, not go up :( Something on my machine must have changed.
Adam Klein
Comment 7 2012-02-10 11:58:13 PST
(In reply to comment #6) > Wait, this isn't right. The score should go down, not go up :( Something on my machine must have changed. Not sure I follow the last two comments. Should we revert r107170 and see if the bot's score improves?
Adam Klein
Comment 8 2012-02-10 12:04:28 PST
(In reply to comment #7) > (In reply to comment #6) > > Wait, this isn't right. The score should go down, not go up :( Something on my machine must have changed. > > Not sure I follow the last two comments. Should we revert r107170 and see if the bot's score improves? Though looking at the test code some more, I'm completely baffled as to how my code could even be running, since afaict this doesn't touch DOM (and thus doesn't touch bindings/v8/) at all.
Ryosuke Niwa
Comment 9 2012-02-10 12:16:22 PST
(In reply to comment #7) > (In reply to comment #6) > > Wait, this isn't right. The score should go down, not go up :( Something on my machine must have changed. > > Not sure I follow the last two comments. Should we revert r107170 and see if the bot's score improves? So... despite the fact it has "ms" as the unit, Dromaeo test results are in "Hz" and we want score to go up instead of go down like other tests. Here, the scores went up in r107170, which means the performance improved. But that makes no sense whatsoever and it's probably to do with my local setup.
Ryosuke Niwa
Comment 10 2012-04-01 17:35:59 PDT
I could not figure out what was going on. This is obsolete now.
Note You need to log in before you can comment on or make changes to this bug.