Some uses of the canvas 2d rendering API could benefit from hardware acceleration. This is a master bug for using OpenGL ES 2.0 as a canvas 2d rendering backend for canvas 2d.
When you say "OpenGL ES 2.0" do you mean OpenGL in general, or are you only interested in embedded devices? There are no ES 2.0 drivers for desktop machines that I know of.
Perhaps this would be better phrased as "GPU-based Hardware Acceleration for Canvas"? Of course that begs the question, why only for Canvas? Wouldn't all 2D rendering in WebKit benefit from hardware acceleration?
(In reply to comment #1) > When you say "OpenGL ES 2.0" do you mean OpenGL in general, or are you only interested in embedded devices? There are no ES 2.0 drivers for desktop machines that I know of. The GPU process used by Chrome (for accelerated compositing and WebGL currently) uses OpenGL ES 2.0 as an interface. It has backends for desktop GL, Mesa and D3D (via ANGLE), it just uses OGLES2 as a "protocol", if you will.
That is just their nomenclature. It would be more accurate to call the "protocol" WebGL, since that's what it is. WebGL is similar to, but not identical to ES 2.0. Or maybe it would be simpler to talk about the class interface being used, which is GraphicsContext3D.
> When you say "OpenGL ES 2.0" do you mean OpenGL in general, or are you only interested in embedded devices? There are no ES 2.0 drivers for desktop machines that I know of. The idea is to code things to OpenGL ES 2.0 and then have a glue layer for desktop GLs (when needed). In Chrome we have such a shim layer (like Stephen said) but for other ports I think the only shim layer needed is a few #defines for functions that have different names between desktop GL and GL ES 2.0 since ES is essentially a subset of what has existed in desktop GL for a while now. If it turns out that there is some feature that isn't possibly to easily use on widely deloyed desktop Open GL then we can just be careful to avoid it. However, I think GL ES 2.0 should serve as a good lowest common denominator to code against. > Perhaps this would be better phrased as "GPU-based Hardware Acceleration for Canvas"? People may want to write a Direct2D or <insert accelerated graphics library here> backend for canvas. That would probably be served better by a separate master bug. > Of course that begs the question, why only for Canvas? Wouldn't all 2D rendering in WebKit benefit from hardware acceleration? Yes, it certainly would. However I think that's somewhat of a separate problem. The most obvious difference is that canvas 2d is an immediate mode API whereas normal rendering of HTML or SVG content is inherently retained. This means that the ideal design for hardware accelerated rendering in general might look different.
I should add that once we have a solid OpenGL backend for canvas it would probably not be too much work to write a GraphicsContext impl using the backend and see how it performs.
(In reply to comment #5) > The idea is to code things to OpenGL ES 2.0 and then have a glue layer for desktop GLs (when needed). In Chrome we have such a shim layer (like Stephen said) but for other ports I think the only shim layer needed is a few #defines for functions that have different names between desktop GL and GL ES 2.0 since ES is essentially a subset of what has existed in desktop GL for a while now. But what you're talking about is exactly what GraphicsContext3D does. It is designed as an API which serves the needs of WebGL. Today it has backends for Mac OpenGL for WebKit, and Mac/Windows/Linux OpenGL for Chrome. It will soon have a backend for D3D via ANGLE and at some point it will no doubt have a true OpenGL ES 2.0 backend for mobile devices. If you ported the 2D Canvas code on top of that, it can be used on all the above back ends. Porting directly to ES 2.0 with glue for desktop OpenGL is just duplicating effort. Seems like a version of GraphicsContext (which is what 2D Canvas uses) on top of GraphicsContext3D would give you what you want and would be easier to write. > > If it turns out that there is some feature that isn't possibly to easily use on widely deloyed desktop Open GL then we can just be careful to avoid it. However, I think GL ES 2.0 should serve as a good lowest common denominator to code against. We've already had to deal with all these diffs in WebGL, as well as D3D diffs, which is why I say WebGL (via GraphicsContext3D) is the better model to use. > > > Perhaps this would be better phrased as "GPU-based Hardware Acceleration for Canvas"? > > People may want to write a Direct2D or <insert accelerated graphics library here> backend for canvas. That would probably be served better by a separate master bug. True. It may be useful to have a separate implementation of GraphicsContext on top of Direct2D, and I agree that would be a separate bug.
GraphicsContext3D is tightly coupled with WebGL and it seems odd to make a canvas 2d backend depend on WebGL. Some ports (perhaps mobile) may want to use a GL-accelerated canvas but not compile WebGL in at all. WebGL is very close to GL ES 2, so switching between the two is not very difficult in terms of amount of code, but I think it's a slight layering violation. Coding to OpenGL ES 2.0 will currently work without any changes or extra glue needed on Chromium on all platforms and on ANGLE. To work on desktop OpenGL everywhere each port has to have a glue header with lines like this: #define glBindFramebuffer glBindFramebufferEXT guarded by the appropriate #ifdefs.
Another (possibly better) option would be to refactor the WebGL specific stuff out of GraphicsContext3D and then just use GraphicsContext3D everywhere. Currently there's some things in GraphicsContext3D (like defensive input validation) that wouldn't be as useful for a canvas 2d backend. I think the refactor will be a bit of work.
(In reply to comment #9) > Another (possibly better) option would be to refactor the WebGL specific stuff out of GraphicsContext3D and then just use GraphicsContext3D everywhere. That sounds like the correct approach.
(In reply to comment #7) > (In reply to comment #5) > > > The idea is to code things to OpenGL ES 2.0 and then have a glue layer for desktop GLs (when needed). In Chrome we have such a shim layer (like Stephen said) but for other ports I think the only shim layer needed is a few #defines for functions that have different names between desktop GL and GL ES 2.0 since ES is essentially a subset of what has existed in desktop GL for a while now. > > But what you're talking about is exactly what GraphicsContext3D does. It is designed as an API which serves the needs of WebGL. Today it has backends for Mac OpenGL for WebKit, and Mac/Windows/Linux OpenGL for Chrome. It will soon have a backend for D3D via ANGLE and at some point it will no doubt have a true OpenGL ES 2.0 backend for mobile devices. If you ported the 2D Canvas code on top of that, it can be used on all the above back ends. > > Porting directly to ES 2.0 with glue for desktop OpenGL is just duplicating effort. Seems like a version of GraphicsContext (which is what 2D Canvas uses) on top of GraphicsContext3D would give you what you want and would be easier to write. We considered many different approaches when we looked at this issue. However, due to Chrome's sandbox model, we don't make any GL or D3D calls directly from the renderer process, so we couldn't use any of the GC3D backends other than the marshalled OpenGL ES 2.0 one anyway. Thus, I don't think GraphicsContext3D would buy us much for Chrome, except a layer of indirection we don't need. I can see how it might make life easier for other ports, though, so it's something we could look at once the WebGL-isms have been factored out. At that point, it should be fairly straightforward to convert the GLES2Canvas code to use it.
(In reply to comment #8) > GraphicsContext3D is tightly coupled with WebGL and it seems odd to make a canvas 2d backend depend on WebGL. Some ports (perhaps mobile) may want to use a GL-accelerated canvas but not compile WebGL in at all. WebGL is very close to GL ES 2, so switching between the two is not very difficult in terms of amount of code, but I think it's a slight layering violation. The only reason GraphicsContext3D is "tightly bound" to WebGL is because that is its only client. We named it GraphicsContext3D purposely to avoid it being simply a WebGL class. > > Coding to OpenGL ES 2.0 will currently work without any changes or extra glue needed on Chromium on all platforms and on ANGLE. To work on desktop OpenGL everywhere each port has to have a glue header with lines like this: > > #define glBindFramebuffer glBindFramebufferEXT > > guarded by the appropriate #ifdefs. I'm not sure why you say that. The only place there are any OpenGL specific calls in in GraphicsContext3DMac, which is the Mac specific version of GraphicsContext3D. One that used Angle would have a different implementation. I'm just talking about using GraphicsContext3D as an interface with whatever backend is needed. If the code were similar enough you could do an implementation with ifdefs for both OpenGL and OpenGL ES, but that may not be the best approach.
There are several cleanups and refactorings planned in the WebGL and GraphicsContext3D implementation over the next month or so, for example https://bugs.webkit.org/show_bug.cgi?id=31564 , https://bugs.webkit.org/show_bug.cgi?id=38619 , and https://bugs.webkit.org/show_bug.cgi?id=38761 . I agree that we should try to factor out WebGL-isms from GraphicsContext3D and eliminate some outstanding layering violations in this code, but while we are trying to clean up and complete the WebGL 1.0 implementation over the next month or so I would really appreciate it if we could avoid code churn due to more large refactorings in GraphicsContext3D. I don't mean to impede progress, so if there is something that is blocking the 2D acceleration effort let's figure out how to do it in a way that doesn't derail the upcoming WebGL 1.0 release.
(In reply to comment #9) > Another (possibly better) option would be to refactor the WebGL specific stuff out of GraphicsContext3D and then just use GraphicsContext3D everywhere. Currently there's some things in GraphicsContext3D (like defensive input validation) that wouldn't be as useful for a canvas 2d backend. I think the refactor will be a bit of work. It would be reasonable to create a WebGLGraphicsContext which does all the extra checks (either as a container class or a subclass) and then remove them from GraphicsContext3D. But I don't think that needs to be done now since it's just an optimization. And as kbr says it would be best if we held off for a while on these kind of changes to GraphicsContext3D. The real question is whether or not any API changes are needed to support the needs of a Canvas backend. I hope not since GraphicsContext3D is nothing more than an OpenGL ES 2.0 API with the GL "objects" replaces by actual C++ objects.