Bug 31232 - [Qt] Windowed Plugins : Implement composited plugins for windowed mode
Summary: [Qt] Windowed Plugins : Implement composited plugins for windowed mode
Status: RESOLVED INVALID
Alias: None
Product: WebKit
Classification: Unclassified
Component: Plug-ins (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC Linux
: P2 Normal
Assignee: Nobody
URL:
Keywords: Qt, QtTriaged
Depends on:
Blocks: 24157
  Show dependency treegraph
 
Reported: 2009-11-07 15:50 PST by Girish Ramakrishnan
Modified: 2014-02-03 03:15 PST (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Girish Ramakrishnan 2009-11-07 15:50:59 PST
Currently, flash does not display in QWebGraphicsView since it's in windowed mode. This is solved by using composite extension for windowed mode.

Fennec patch - https://bugzilla.mozilla.org/show_bug.cgi?id=442109

Pure Qt implementation - http://git.forwardbias.in/?p=webkit/xembedoffscreen.git;a=summary
Comment 1 Girish Ramakrishnan 2009-11-10 01:22:00 PST
I have a working implementation outside WebKit: http://git.forwardbias.in/?p=webkit/xembedoffscreen.git;a=summary. Events don't work only with flash (works with Qt and Gtk widgets), not sure why.
Comment 2 Mark Renouf 2009-11-10 10:15:03 PST
Can someone describe in a bit more detail what this bug seeks to implement, vs. what has been implemented in #18831 ?
Comment 3 Girish Ramakrishnan 2009-11-10 10:38:21 PST
(In reply to comment #2)
> Can someone describe in a bit more detail what this bug seeks to implement, vs.
> what has been implemented in #18831 ?

Sure.

Flash (Netscape plugins really) works in two modes - windowed and windowless. In the windowed mode, it expects a native window. In the windowless mode, it can draw itself into a pixmap. By default, the mode is windowed. One needs to use wmode=transparent or opaque (Flash specific) to make it windowless.

Whether the browser displays a plugin in windowed or windowless mode is a decision made by the plugin. It cannot be enforced by the browser (In theory, a browser can advertise only windowless mode but this causes Flash to crash).

In Qt, we have two different 'webkit' views 
1. QWebView, a widget. It is possible to XEmbed into a QWebView. 
2. We also have a QGraphicsWebView - this is an item in a 2d scene (QGraphicsScene). Items can be transformed. So, it's not possible to use XEmbed in this case. As a result, we only support windowless mode in a QGraphicsWebView.

Now to the problem - When using youtube, the flash content enforces windowed mode. As a result, youtube videos don't appear in QGraphicsWebView.

Solutions: 
1. We can either use XComposite/XDamage to get the contents of the plugin into a Picture and render it into the QGraphicsWebView. (This bug addresses this solution). (This approach is used in Fennec)
2. Inject wmode=transparent or similar into the dom/pass it to the plugin when instantiating it. Since, we support windowless mode, youtube will appear in the QGraphicsWebView.

We haven't decided yet which of the above is 'elegant'. This is still research.
Comment 4 Simon Hausmann 2009-11-10 12:59:33 PST
(In reply to comment #1)
> I have a working implementation outside WebKit:
> http://git.forwardbias.in/?p=webkit/xembedoffscreen.git;a=summary. Events don't
> work only with flash (works with Qt and Gtk widgets), not sure why.

That's surprisingly few lines of code - nice :)

Wrt the events: Currently you're sending the events via XSendEvent. Given flash's use of Gtk I would assume that Gtk/Gdk receives the events and passes them on to Flash as GdkEvents. It seems they get stuck somewhere, right?

What you could try is the following: Instead of the round trip to the X server and the translation through Gtk/Gdk, what happens if you send the events "directly" just like you would send them to a windowless plugin? It might be quicker and with a bit of luck it might work, too ;-)
Comment 5 Girish Ramakrishnan 2009-11-10 20:34:10 PST
(In reply to comment #4)
> What you could try is the following: Instead of the round trip to the X server
> and the translation through Gtk/Gdk, what happens if you send the events
> "directly" just like you would send them to a windowless plugin? It might be
> quicker and with a bit of luck it might work, too ;-)

Ah, brilliant suggestion as always. This actually works. Just pushed a change to my git repo. So, I will try merging to webkit now.

Since, the code is very different from the embed container code, I am thinking we will add this as a separate file PluginCompositedQt.[cpp|h]. This also makes it easy to ifdef out the whole thing. To start with, we can make PluginComposited a WebCore Widget but in the long run, I guess we can remove it too. In fact, we should be able to reuse (in theory) the windowless event dispatching and the composited mode event dispatching. I have to check if refactoring that portion is worth the lesser readability though.
Comment 6 Girish Ramakrishnan 2009-11-11 04:50:56 PST
http://gitorious.org/~girish/qtwebkit/girishs-clone/commits/composited_plugins has a half-broken implementation, but youtube can now be flipped in QGVLauncher :-)

I am going to disappear for a while (am working on this on my own time), I will be back next week. If someone feels like cleaning it up ping me on irc (girishr).
Comment 7 Girish Ramakrishnan 2009-11-11 05:00:39 PST
Here's an (incomplete) TODO list:
1. Check if we can just use QWidget::x11Event instead of converting Qt events.
2. Fix Event handling
3. Track old event filters
4. Transparent plugins does not work yet
5. We can just make PluginCompositedQt a QX11EmbedContainer

Unfixable/Don't know how to handle cases below:
1. Popups shown by plugin
2. Cursor changes by plugin
3. Dialogs shown by plugin
Comment 8 Girish Ramakrishnan 2009-12-02 01:58:20 PST
Per my understanding, the approach of using xcomposite was chosen by fennec, only because they didn't have flash 10 on ARM (flash 10 added support for windowless mode). 

See https://bugzilla.mozilla.org/show_bug.cgi?id=442109#c10 and https://bugzilla.mozilla.org/show_bug.cgi?id=442109#c11
Comment 9 Girish Ramakrishnan 2009-12-02 01:59:27 PST
See also #32059
Comment 10 Girish Ramakrishnan 2009-12-11 07:35:37 PST
There's a Qt3 based composite implementation.
http://ktown.kde.org/~fredrik/compositewrapper.cpp
http://ktown.kde.org/~fredrik/compositewrapper.cpp
Comment 11 Tor Arne Vestbø 2010-03-10 06:37:38 PST
Please follow the QtWebKit bug reporting guidelines when reporting bugs.

See http://trac.webkit.org/wiki/QtWebKitBugs

Specifically:

  - The 'QtWebKit' component should only be used for bugs/features in the
    public QtWebKit API layer, not to signify that the bug is specific to
    the Qt port of WebKit

      http://trac.webkit.org/wiki/QtWebKitBugs#Component

  - Add the keyword 'Qt' to signal that it's a Qt-related bug

      http://trac.webkit.org/wiki/QtWebKitBugs#Keywords
Comment 12 Mozhaev Grigory 2012-12-15 11:55:26 PST
Still no implementation in QtWebkit for this bug, no flash in any videos of  www.bbc.co.uk
Comment 13 Jocelyn Turcotte 2014-02-03 03:15:55 PST
=== Bulk closing of Qt bugs ===

If you believe that this bug report is still relevant for a non-Qt port of webkit.org, please re-open it and remove [Qt] from the summary.

If you believe that this is still an important QtWebKit bug, please fill a new report at https://bugreports.qt-project.org and add a link to this issue. See http://qt-project.org/wiki/ReportingBugsInQt for additional guidelines.