Bug 29813 – [WinCairo] 3D CSS Transforms are not implemented in WebKit for Windows

• Component: Layout and Rendering
• Status: RESOLVED FIXED

[Brent Fulgham]

This bug is intended to track conversation about possible ways to implement 3D CSS transformation in the redistributable WebKit for Windows (WinCairo).  Some of these ideas may be useful for other platforms (perhaps including the official Apple Windows port), but are initially listed here so as not to clutter Bug 27314 (which deals with the official port.)

The main goals of this task are:
(1) Provide very similar behavior and look to the implementation under Mac OS X.
(2) Take advantage of 3D hardware acceleration when possible, with some fallback for systems that lack this functionality.
(3) Minimize differences from the main WebKit source code so that a 3D accelerated WebKit to reduce inadvertent behavior changes and avoid 'forking' logic, thereby increasing the effort to retain feature parity with the official Apple release.

As far as other platform implementations, I have only heard that the Chromium port has some interest in getting 3D CSS running on Windows/Linux, it is not a high priority due to the complexity of sharing a single piece of hardware with multiple rendering processes.  They may attempt to provide a software implementation but no one is actively working on this (or planning to in the near future).  I have not heard of any progress on the Linux or other platforms in implementing this feature.

Possible Architecture
----------------
I took a little time to track down a suitable rendering option to support the 2D drawing we generally need, as well as allowing easy compositing of 3D information.  The following projects have software that seems likely to be useful:

(1) Clutter:  The first obvious target is Clutter (http://www.clutter-project.org/), which implements a compositing and animation engine on top of OpenGL. It has some preexisting bindings to Cairo for use in 2D rendering, as well as some earlier work integrating with WebKit to allow rendering of HTML onto OpenGL surfaces.

Unfortunately, this library is very hard to build on Windows due to its many build and runtime requirements (glib, pango, gobject) and build environment (mingw32 support, or use an external 'OAH' build
(https://launchpad.net/oah/+download) system.)  Windows development seems sort of half-hearted at the moment, and I found it hard to get answers to various build environment questions.

(2) Cairo/OpenGL implementations: I then saw that a couple of ports of Cairo to sit on OpenGL.  One of them uses an implementation of OpenVG (a 2D rendering API specification from the OpenGL/Khronos people)  See http://ivanleben.blogspot.com/2007/07/shivavg-open-source-ansi-c-openvg.html for the original blog posting, and http://sourceforge.net/projects/shivavg/ for the project page.  There are a handful of people who have modified Cairo to use Shiva for rendering (http://lists.ximian.com/pipermail/moonlight-list/2009-July/000561.html), including our own Alp Toker.

Given that Shiva uses OpenGL internally, it should be possible to make use of it for performing the 3D transformations needed, then composite with the rest of the web page rendering tree.

(3) Direct3D:  I also looked into using Direct3D.  I am not a big fan of this approach because: (a) Any code written for Direct3D is limited to the Windows platform; any OpenGL implementation can be shared to some extent between the various ports, and (b) A Direct3D implementation can't be leveraged to provide an implementation of the "3d canvas" extension (see, e.g., Bug 29010), which basically creates an OpenGL context that can be programmed from JavaScript.

Still, Direct3D is probably the best supported option on Windows and has a good likelihood of being present on most Windows boxes.  I spent a few hours playing around with this, but ran into trouble when trying to figure out how to composite any Direct3D data into the main rendered content.

(4) Direct2D and Direct3D:  Finally, I considered trying to do something like target a Direct2D backend for rendering the main frame context, while using Direct3D for any 3D features.  This might work, but Direct2D won't be available on Windows XP and is not yet a released product.

(5) WPF: The Windows Presentation Foundation layer could perhaps be used for its rendering and compositing features.  WPF is hardware accelerated, and will be present on any Windows machine running .Net 3.0 or newer.  However, I am troubled by the idea of dispatching from the core rendering kernel of WebKit across an interop layer to .NET, run some layout or other drawing logic in MSIL or whatever, dispatch across .Net back into Direct3D, and then into hardware.  The rendering path is some of the most carefully tuned code in WebKit; regions of it are constantly being rewritten to reduce extra object copies, allocations, and even virtual method overhead.  Shoving a big interpreted runtime in here (even if JIT compilation helps) seems likely to be a bad thing.

Initial Thoughts
------------
I have to admit to being a bit biased towards OpenGL.  I am familiar with it from my 'real' job, and since it runs on the Mac, Windows, and Linux it has the most opportunity for synergy with the other ports (read: less code to write myself).  I don't know Direct3D at all, though it wasn't that hard to play with in my initial exploration.

It would be great to limit the scope of any 3D work to avoid breaking existing functionality.  However, it might not be possible to implement this feature properly without rethinking the rendering architecture to best take advantage of platform features.  My hope would be to limit changes to platform-specific files in the "platform/graphics/win" and "rendering/", rather than attempting to bypass large regions of the existing rendering code path.  I think this will only create headaches and incompatibilities.  For example, looking at the "rendering/" code, button painting (among other things) is implemented in "RenderThemeWin.cpp" as a single function that breaks the rendering into a series of Win32 draw commands.  It seems like this (and perhaps a handful of other) implementation files could be duplicated and implemented using something besides Win32 commands without creating too much variation from the core Windows implementation.  To continue this example, "paintButton" could be implemented in terms of some Win-specific routine, rather than dispatching to the generic drawControl method.

The approach I was forming in my mind was to get Cairo up and running on OpenGL (probably using something like OpenVG), then bypass VG to interact with OpenGL directly for the 3D effects.

I don't think any solution that involves replacing large portions of WebKit logic with platform-specific routines is a good option for the following reasons:

1.  As I mentioned earlier, much of the widgets we want to throw on the screen don't necessarily correspond to canned widgets from the underlying platform.
2.  The smaller the variation between our port and the official Apple port, the less variations in behavior, self-introduced bugs, and functionality lag we can expect.  If our rendering path bypasses significant regions of the core WebKit port, new features or behaviors introduced upstream will become new logic we will have to duplicate in our custom path.
3.  I think violating pixel-identical rendering is a bad idea.  I want to still *be* WebKit, not a "similar" product.  :-)
4.  Basing a 3D CSS implementation on Direct3D/Direct2D/WPF doesn't do much for providing an implementation supporting the <canvas3d> tag (which exposes OpenGL drawing commands to JavaScript).

[Brent Fulgham]

Critical areas of the source base to consider are:

* WebCore/rendering (specifically the *Win.cpp implementations).
* WebCore/platform/graphics/mac/GraphicsLayerCA.mm must be ported to the Windows platform (perhaps as platform/graphics/win/GraphicsLayerWin)

Of special concern is WebCore/rendering/RenderLayerCompositor.cpp, which does the work of compositing the various elements onto the final surface for display.  Its implementation makes use of a call "atachRootGraphicsLayer", which in turn calls the WebHTMLView classes "attachRootLayer" method, which in turn uses the CoreAnimation "addSubview" method.

[Brent Fulgham]

Google chrome apparently has a software renderer implementation of 3d CSS (http://src.chromium.org/viewvc/chrome/trunk/src/o3d/).  The original plugin appears to provide full hardware acceleration via direct 3d, though it was designed to work outside of WebKit, and so will probably not reveal too much insight into integrating 3D into the overall rendering path.

[Brent Fulgham]

Direct2D is part of the current DirectX install, and will function under Windows Vista.  We might need to implement this feature along the lines of Apple's Tiger/Leopard/Snow Leopard feature distinctions and limit this to Vista and beyond.

(1) http://blogs.msdn.com/tmulcahy/ Shows some interesting effects achieved using DIrect2D, many of which fall in line with the needs of the 3D CSS transform.
(2) http://blogs.msdn.com/greg_schechter/archive/2007/03/30/parallaxui_5F00_escaping_5F00_flatland.aspx includes a few interesting 3D CSS-style transforms using the unfortunate WPF system, but obviously are based on the underlying implementation.

My preference is still to base the design on OpenGL, especially given the progress on Cairo-on-OpenGL, which would provide a Linux/Windows shared core implementation.

[Brent Fulgham]

Given the progress on Bug 27314, it looks like we should be taking the Direct3D track for implementing this.

[Peter Hewat]

Hello Brent,
It has been interesting reading your thoughts regarding this issue.
I too am biased towards OpenGL for the simple reason that it is multi-platform whereas Direct3D is Windows specific.
My worry is that the Linux platform will trail behind regarding this issue. Has there been any progress on the Linux front?

[James Star]

Hello, I am using Safari for Windows version 5.0. I am using a webkit 3d transformation, perspective rotation etc... and placing a form inside a HTML section tag, and I am uncertain as to why but the html form disappears.

Is this issue related to this post at all?

Regards,
James

[Simon Fraser (smfr)]

(In reply to comment #6)
> Hello, I am using Safari for Windows version 5.0. I am using a webkit 3d transformation, perspective rotation etc... and placing a form inside a HTML section tag, and I am uncertain as to why but the html form disappears.
> 
> Is this issue related to this post at all?

No, Apple's Safari on Windows does not use Cairo. Please file a separate bug with your testcase.

[Brent Fulgham]

Alex and Peavo: I think this feature is now implemented. Can we close this old Bug now?

(In reply to comment #8)
> Alex and Peavo: I think this feature is now implemented. Can we close this
> old Bug now?

Yes, I believe so.