Bug 140745 – [GTK] Support direct Python web extension loading

• Status: NEW
• Component: WebKitGTK

[Adrian Perez]

Using WebKitGTK+ from Python is supported thanks to the GObject-Introspection
support, but some parts of the API (mainly, the DOM bindings) have to be used
from the WebProcess when using WebKit2 in a Web Extension. Right now we only
support direct loading of Web Extensions implemented in C: even when it is
possible to run Python scripts using the libpython C API from a Web Extension,
each application would need to provide a Web Extension for doing so, leading to
duplicate boilerplate code.

Provided that using WebKitGTK+ embedded in GTK+ applications written in Python
is quite popular, it would be good to provide a loader for Python Web
Extensions.

[Adrian Perez]

I have made a small experiment, and supporting loading a Python module
as a WebExtension can be done with reasonably small amount of code. As
a matter of fact, using a small C shim extension to load a Python module
works fine:

  https://github.com/aperezdc/webkit2gtk-python-webextension-example

The code in the above repository creates a PyObject that represents the
WebKitWebExtension, and the user_data GVariant. It uses the PyGObject
API, so the created objects are wrapped in the same familiar way as the
rest of Python bindings.

It would be great to have this functionality in WebKitGTK+ itself, but
adding PyGobject, and libpython as dependencies does not sound great.
In my opinion the way to go would be to provide a C shim extension (in
a well-known filesystem location), which would be the component linked
to PyGObject and libpython (so libwebkit2gtk would not link to them),
and install it along with WebKitGTK+, which would provide a new API
function:

  bool webkit_web_context_set_python_web_extension_path(
            WebKitWebContext *web_context, const gchar *path,
            GError **error);

The function receives the path to a single .py file which is to be
loaded as a Python Web Extension, and it returns TRUE on success. The
GError would be set with one of the following error codes:

  enum WebKitPythonWebExtensionError {
       /* Python support not built in, or the C shim cannot be loaded */
       WEBKIT_PYTHON_WEB_EXTENSION_ERROR_UNSUPPORTED,

       /* Python module can not be imported. */
       WEBKIT_PYTHON_WEB_EXTENSION_ERROR_IMPORT,

       /* The Python intialization function raised an exception. */
       WEBKIT_PYTHON_WEB_EXTENSION_ERROR_EXCEPTION,
  };

You may be wondering why allowing only a single Python module as a web
extension. The reason is that Python (CPython, in particular) does not
support running multiple concurrent VM states well, so all the scripts
would be loaded in the same Python VM anyway. Provided that Web Extensions
are used by programs embedding WebKitGTK+, the people developing those
can accomodate to provide just a single Python module, or they can load
themselves additional Python modules using the mechanisms provided by
Python itself, and initialize them passing the WebKitWebExtension from
the top-level Python module loaded by the C shim provided by WebKitGTK+.

How does this plan sound?

[Adrian Perez]

Actually, after checking a bit how web extension loading is done,
we don't even need to change the API, it would be possible to modify
WebGtkExtensionManager::initialize() to also search for '*.py' files
in the web extensions directory. If any is found, then it would load
the Python web extension loader, which in turn would load the '*.py'
files and call their "initialize(extension, user_data)" functions.

The Python web extension loader would be a small shared object, at
"${prefix}/lib/webkit2gtk-4.0/extension-loader/libloader-python.so"
for example, and that would be linked to libpython and all the other 
dependencies which are not desirable for libwebkit2gtk.so — this way
we would not force users to have Python installed. Which loaders are
built would be a compile-time option. At runtime, if the script loader
shared object cannot be used, extensions written in the corresponding
language would not be loaded instead of failing (example: libpython
is not installed). This would allow distributions to build WebKitGTK+
with support for the Python script loader, without forcing users to
alwayts install libpython.

Note that all Python scripts still would be loaded in the same
Python VM — for the reasons mentioned in my previous comment. Loaders
for scripts in other languages could isolate each script in its own
VM (this is very doable in the case of Lua, for example).

Note that script loaders for languages other than Python could be
added using the same machinery (Lua, JavaScript, etc.).

[Philip Chimento]

May I suggest adding a loader for GJS (using GObject introspection) as part of the reference implementation? GJS has been Gnome's first-class language of choice for several years now [1]. Also, being able to write web extensions in Javascript would lower the bar for people who primarily know web development but are wrapping some web application in a WebKitGTK container.

[1] http://treitter.livejournal.com/14871.html

[Adrian Perez]

(In reply to comment #3)
> May I suggest adding a loader for GJS (using GObject introspection) as part
> of the reference implementation? GJS has been Gnome's first-class language
> of choice for several years now [1]. Also, being able to write web
> extensions in Javascript would lower the bar for people who primarily know
> web development but are wrapping some web application in a WebKitGTK
> container.
> 
> [1] http://treitter.livejournal.com/14871.html

Sure, there is nothing wrong in trying to support JS web extensions, but
it looks better to me to track support for that separately. I have just
created a new bug report for the JS web extension support:

  https://bugs.webkit.org/show_bug.cgi?id=148711

For the moment I am more interested in implementing Python support first,
and keeping an eye in making it in a way that allows adding support for
other languages later on.

[Cédric Bellegarde]

This really need to be fixed? Any news on this?

[Michael Catanzaro]

(In reply to comment #5)
> This really need to be fixed?

I don't think so. We can't provide special APIs for every language under the sun. In particular, surely we do not want WebKit to depend on PyGObject (or gjs). That's not to say that the Python loader is not valuable, but I think it should be maintained separately from the WebKit project. And Adrian's GitHub repo seems as fine a place as any.