The json results generated by the Animometer benchmark when ran with the run-benchmark tool have two Animometer': {metrics': {Score': [Geometric']}, tests': {Animometer': {metrics': {Score': [Geometric']}, tests': {Canvas Arcs': {metrics': {Score': {current': [1425.4671861375848]}}}, Canvas Lines': {metrics': {Score': {current': [6041.67557789635]}}}, Design': {metrics': {Score': {current': [120.42884422240773]}}}, Focus': {metrics': {Score': {current': [106.53534015865043]}}}, Images': {metrics': {Score': {current': [141.71080576796632]}}}, Leaves': {metrics': {Score': {current': [986.0850870259096]}}}, Multiply': {metrics': {Score': {current': [1080.5982243695066]}}}, Paths': {metrics': {Score': {current': [4659.013255111744]}}}, Suits': {metrics': {Score': {current': [1137.5734845930879]}}}}}}} Notice how there is a main aggregated geometric result with name Animometer, which contains inside another result of the same type and name. This is wrong, see the output of read-results: $ Tools/Scripts/run-benchmark --read-results-json animometer.result Animometer:Score:Geometric: 250pt stdev=0.0% Animometer:Score:Geometric: 250pt stdev=0.0% Canvas Arcs:Score: 256pt stdev=0.0% Canvas Lines:Score: 967pt stdev=0.0% Design:Score: 50.0pt stdev=0.0% Focus:Score: 255pt stdev=0.0% Images:Score: 208pt stdev=0.0% Leaves:Score: 103pt stdev=0.0% Multiply:Score: 336pt stdev=0.0% Paths:Score: 915pt stdev=0.0% Suits:Score: 184pt stdev=0.0% Notice here how two entries with "Animometer:Score:Geometric: 250pt " appear
Created attachment 295180 [details] Patch
After this patch the contents of the JSON file are (without the debugOutput data): Animometer': {metrics': {Score': [Geometric']}, tests': {Canvas Arcs': {metrics': {Score': {current': [251.94623218197282]}}}, Canvas Lines': {metrics': {Score': {current': [871.206693453393]}}}, Design': {metrics': {Score': {current': [47.84977960868452]}}}, Focus': {metrics': {Score': {current': [211.9234170550923]}}}, Images': {metrics': {Score': {current': [199.28169278994972]}}}, Leaves': {metrics': {Score': {current': [101.77153542904863]}}}, Multiply': {metrics': {Score': {current': [335.41278407113873]}}}, Paths': {metrics': {Score': {current': [898.6858128956474]}}}, Suits': {metrics': {Score': {current': [181.91975476813016]}}}}} $ Tools/Scripts/run-benchmark --read-results-json animometer.result Animometer:Score:Geometric: 238pt stdev=0.0% Canvas Arcs:Score: 252pt stdev=0.0% Canvas Lines:Score: 871pt stdev=0.0% Design:Score: 47.8pt stdev=0.0% Focus:Score: 212pt stdev=0.0% Images:Score: 199pt stdev=0.0% Leaves:Score: 102pt stdev=0.0% Multiply:Score: 335pt stdev=0.0% Paths:Score: 899pt stdev=0.0% Suits:Score: 182pt stdev=0.0% And run-benchmark produces:
Comment on attachment 295180 [details] Patch This isn’t quite right. Animometer allows multiple suites to be ran, and the one we’re reporting right now is the MotionMark suite. I’ll go talk with Jon to figure out what to do here.
(In reply to comment #3) > Comment on attachment 295180 [details] > Patch > > This isn’t quite right. Animometer allows multiple suites to be ran, and the > one we’re reporting right now is the MotionMark suite. I’ll go talk with > Jon to figure out what to do here. Perhaps the MotionMark suite is wrongly named as Animometer? What about something like this? --- a/PerformanceTests/Animometer/resources/runner/tests.js +++ b/PerformanceTests/Animometer/resources/runner/tests.js @@ -29,7 +29,7 @@ var Suite = function(name, tests) { var Suites = []; -Suites.push(new Suite("Animometer", +Suites.push(new Suite("MotionMark", [ { url: "master/multiply.html",
We do need to rename the directory to MotionMark.
(In reply to comment #5) > We do need to rename the directory to MotionMark. So the benchmark itself is named MotionMark, but the current test-suite that is ran by default is named Animometer?
(In reply to comment #6) > (In reply to comment #5) > > We do need to rename the directory to MotionMark. > > So the benchmark itself is named MotionMark, but the current test-suite that > is ran by default is named Animometer? No, it’s more that both needs to be renamed to MotionMark. MotionMark is both a test suite that runs animation performance tests as well as a specific test suite ran by default so the fact it’s nested is actually correct.
The patches <http://trac.webkit.org/changeset/210459> <http://trac.webkit.org/changeset/210463> rename the directory, the patch and the plan files form Animometer to MotionMark.
Created attachment 299574 [details] Fixes this
Now that the remaining tests have been moved to PerformanceTests/Animation, we can safely make this change.
Comment on attachment 299574 [details] Fixes this Attachment 299574 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/2939035 New failing tests: media/modern-media-controls/tracks-support/tracks-support-show-panel-fullscreen.html
Created attachment 299577 [details] Archive of layout-test-results from ews114 for mac-elcapitan The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews114 Port: mac-elcapitan Platform: Mac OS X 10.11.6
Comment on attachment 299574 [details] Fixes this View in context: https://bugs.webkit.org/attachment.cgi?id=299574&action=review > Tools/ChangeLog:9 > + since the remaining non-MotionMark tests are moved to PerformanceTests/Animation. What are the non-MotionMark tests which are moved to PerformanceTests/Animation? Under this directory I see the same set of files unchanged for a long time.
(In reply to comment #13) > Comment on attachment 299574 [details] > Fixes this > > View in context: > https://bugs.webkit.org/attachment.cgi?id=299574&action=review > > > Tools/ChangeLog:9 > > + since the remaining non-MotionMark tests are moved to PerformanceTests/Animation. > > What are the non-MotionMark tests which are moved to > PerformanceTests/Animation? Under this directory I see the same set of files > unchanged for a long time. content-animation tests. But I guess they aren't really the ones that used to be in MotionMark's directory that weren't part of the MotionMark suite.
Committed r211202: <http://trac.webkit.org/changeset/211202>