Bug 166055 – MotionMark / Animometer gives miss-leading results on browsers with prefixed CSS Transforms

• Status: NEW
• Component: Tools / Tests

[Carlos Alberto Lopez Perez]

I was bisecting an issue that was reported to us related to WebKitGTK+ 2.8 obtaining much better results in MotionMark <http://browserbench.org/MotionMark/> than WebKitGTK+ 2.14.

The issue ended being that WebKitGTK+ 2.8 shipped with prefixed CSS Transforms and >= 2.10 shipped with un-prefixed ones.
Meanwhile MotionMark seems to only support the un-prefixed version of the CSS Transforms.

This are the results of WebKitGTK on a laptop on a WebKit build at:

WebKit r181832:
          Animometer:Score:Geometric: 136pt
                    Canvas Arcs:Score: 233pt
                    Canvas Lines:Score: 822pt
                    Design:Score: 62.8pt
                    Focus:Score: 101pt
                    Images:Score: 1.00pt
                    Leaves:Score: 213pt
                    Multiply:Score: 369pt
                    Paths:Score: 832pt
                    Suits:Score: 203pt


WebKit r181831:
          Animometer:Score:Geometric: 373pt
                    Canvas Arcs:Score: 240pt
                    Canvas Lines:Score: 916pt
                    Design:Score: 76.8pt
                    Focus:Score: 46.4pt
                    Images:Score: 220pt
                    Leaves:Score: 8490pt
                    Multiply:Score: 579pt
                    Paths:Score: 853pt
                    Suits:Score: 195pt


On r181832 <https://trac.webkit.org/changeset/181832> which is the revision that removes the prefix for CSS Transforms, all animations are shown correctly.

On r181831 the result is clearly miss-leading. The third animation and the animation with the letters "design" are not correctly shown on the screen.
But the result given to the user is much better.

I think MotionMark/Animometer needs to either:

  * Gain compatibility with old browsers still using prefixed CSS transforms.
  * Be more robust and detect that the animations are actually not happening as expected, therefore it shouldn't increment the performance counter as it does.

[Michael Catanzaro]

Question: that ENABLE_3D_TRANSFORMS is enabled on our bots in FeatureList.pm, but disabled and unsupported for end users of all CMake ports in WebKitFeatures.cmake. Is there a reason for this discrepancy? It does not seem like it should be an experimental feature anymore if it's been unprefixed for nearly years?

[Simon Fraser (smfr)]

(In reply to comment #1)
> Question: that ENABLE_3D_TRANSFORMS is enabled on our bots in
> FeatureList.pm, but disabled and unsupported for end users of all CMake
> ports in WebKitFeatures.cmake. Is there a reason for this discrepancy? It
> does not seem like it should be an experimental feature anymore if it's been
> unprefixed for nearly years?

A port that isn't able to render 3D transforms might ship with ENABLE_3D_TRANSFORMS turned off.

[Michael Catanzaro]

So it sounds like maybe we should enable this feature (Carlos)...? In a separate bug?

[Carlos Alberto Lopez Perez]

(In reply to comment #3)
> So it sounds like maybe we should enable this feature (Carlos)...? In a
> separate bug?

Are you sure its not already enabled by default?

I think its enabled on a per-port basis:

$ git grep ENABLE_3D_TRANSFORMS Source/cmake/Options*
Source/cmake/OptionsEfl.cmake:WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_3D_TRANSFORMS PUBLIC ON)
Source/cmake/OptionsGTK.cmake:WEBKIT_OPTION_DEPEND(ENABLE_3D_TRANSFORMS ENABLE_OPENGL)
Source/cmake/OptionsGTK.cmake:WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_3D_TRANSFORMS PRIVATE ON)
Source/cmake/OptionsMac.cmake:WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_3D_TRANSFORMS PRIVATE ON)
Source/cmake/OptionsWin.cmake:WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_3D_TRANSFORMS PUBLIC ON)

[Michael Catanzaro]

You're right, I forgot to check.