Summary: | Dromaeo perf-tests results are wrong | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Kentaro Hara <haraken> | ||||||||||
Component: | Tools / Tests | Assignee: | Kentaro Hara <haraken> | ||||||||||
Status: | RESOLVED INVALID | ||||||||||||
Severity: | Normal | CC: | abarth, benjamin, morrita, rniwa, webkit.review.bot | ||||||||||
Priority: | P2 | ||||||||||||
Version: | 528+ (Nightly build) | ||||||||||||
Hardware: | Unspecified | ||||||||||||
OS: | Unspecified | ||||||||||||
Bug Depends on: | 81142 | ||||||||||||
Bug Blocks: | |||||||||||||
Attachments: |
|
Description
Kentaro Hara
2012-03-13 01:33:07 PDT
Created attachment 131576 [details]
Patch
Comment on attachment 131576 [details]
Patch
Please consider to keep using "run/s" since it's stable.
I tried ms. But making them resulted unexpected (and typically longer) running time, especially we cannot assume underlying environment.
Also, I guess |runStyle| could be specified externally via URL query string.
Please do it if possible. Since this Dromaeo source is copied from upstream. If there is any obvious errors there,
I encourage you to upstream the fix.
(In reply to comment #2) > (From update of attachment 131576 [details]) > Please consider to keep using "run/s" since it's stable. > I tried ms. But making them resulted unexpected (and typically longer) running time, especially we cannot assume underlying environment. This is the reason why I removed 'if (runStyle === "runs/s")' part. Now the running time does not change between run/s and ms. > Also, I guess |runStyle| could be specified externally via URL query string. > Please do it if possible. Let me try it. > Since this Dromaeo source is copied from upstream. > If there is any obvious errors there, > I encourage you to upstream the fix. If the patch looks OK, I'll try it. But it will take time, and I would like to land the patch into WebKit first. The current Dromaeo results have been preventing me from evaluating performance optimization patches. Created attachment 131582 [details]
Patch
Created attachment 131599 [details]
Patch
Created attachment 131600 [details]
Patch
Comment on attachment 131600 [details] Patch Clearing flags on attachment: 131600 Committed r110559: <http://trac.webkit.org/changeset/110559> All reviewed patches have been landed. Closing bug. I think this broke most of the Dromaeo tests on WebKit-perf: http://webkit-perf.appspot.com/ They all report a time of 0 now. Comment on attachment 131600 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=131600&action=review > PerformanceTests/Dromaeo/resources/dromaeorunner.js:16 > + median += 1000 / item.median; This makes no sense. The sum of medians isn't the median of all samples. What we need to do is to fix https://bugs.webkit.org/show_bug.cgi?id=78303. We don't want to invent our own score for Dromaeo. |