Bug 113663 – Missing APIs in webkit2gtk (compared to webkit1)

• Status: RESOLVED
• Resolution: INVALID
• Component: WebKitGTK

[Ben Boeckel]

As part of porting uzbl[1] to WebKit2, I've found a number of APIs are missing compared to WebKit1. I was wondering what plans there might be for them and whether they'd be possible again:

  - the SoupSession in inaccessible which was used for:
    * send events on cookie add/delete so that plugins could filter and persist cookies between instances;
    * a command to authenticate to a site;
    * external authentication dialog handling (e.g., scripted auth support);
    * custom proxy handling (I'm planning on using domain sockets rather than TCP sockets in the future);
  - access to frames. There are a few things this blocks:
    * support for issuing JS commands to the currently focused frame (to avoid CORS errors);
    * printing the currently focused frame;
    * commands manipulating frame focus, listing frames, etc.;
  - access to webkit's scrollbars (uzbl has a scroll command for keybindings);
  - a comparable API to WebKitWebPluginDatabase for:
    * refreshing the plugin list;
    * toggling specific plugins (and a property in WebKitWebPlugin for whether it is enabled);
  - a non-async API for plugin queries;
  - a call to remove all web databases used by webkit;
  - web database management (WebKitSecurityOrigin);
  - access to webkit's spellchecking;
  - loading arbitrary content into a frame (webkit_web_frame_load_string);
  - unsetting security policies for schemes (though an API like WebKit1 with enum flags rather than an array of functions would be better anyways);
  - access to the actual JS context of the page (webkit_web_view_get_javascript_global_context returns an isolated context);
  - a function to get a WebKitHitTestResult at an arbitrary point on the page (uzbl has support for emulating clicks, so "last hover target" isn't necessarily useful);
  - setting the zoom type (content or text only);
  - getting the current page's encoding;
  - get/set the total application cache maximum size;
  - a comparable API to WebKitViewportAttributes;
  - access to the DOM of a page/frame;
  - the following properties:
    * on WebKitWebSettings:
      - auto-resize-window,
      - enable-universal-access-from-file-uris (ignorable if unsetting security policies is possible),
      - enable-file-access-from-file-uris (ignorable if unsetting security policies is possible),
      - enable-display-of-insecure-content (there is only a signal that it happened),
      - enable-running-of-insecure-content (same as display variant),
      - enable-scripts,
      - enable-dom-paste,
      - auto-shrink-images,
      - respect-image-orientation,
      - enable-spatial-navigation,
      - editing-behavior,
      - enforce-96-dpi,
      - minimum-font-size,
      - minimum-logical-font-size,
      - enable-accelerated-compositing (is this always true in WebKit2?),
      - enable-css-shaders,
      - enable-media-stream,
    * on WebKitWebInspector:
      - javascript-profiling-enabled,
      - timeline-profiling-enabled,
    * on WebKitWebView:
      - transparent,
      - editable.

If I'm missing something in WebKit2 that does the same thing, I may have missed it. I've been over both sets of docs a few times and still find new things every now and then. If there is no plans to support a feature, the only ones which are 100% required for uzbl to work is access to the scrollbars, SoupSession, and page's JavaScript context.

For further tracking of specific items, creating a new bug is probably preferable (rather than polluting this bug). However, making one bug for each from the start seems overkill if I've missed things and half are just going to be closed right away.

[1]http://uzbl.org

[Carlos Garcia Campos]

(In reply to comment #0)
> As part of porting uzbl[1] to WebKit2, I've found a number of APIs are missing compared to WebKit1. I was wondering what plans there might be for them and whether they'd be possible again:

Thanks! I'll reply to some of them.

>   - the SoupSession in inaccessible which was used for:

This is simply not possible, because the network is in the web process. However, we expose in the UI process API for several network features.

>     * send events on cookie add/delete so that plugins could filter and persist cookies between instances;

http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitCookieManager.html#WebKitCookieManager-changed

>     * a command to authenticate to a site;

HTTP authentication should just work by default.

>     * external authentication dialog handling (e.g., scripted auth support);

We plan to add API to allow handling HTTP auth request.

>     * custom proxy handling (I'm planning on using domain sockets rather than TCP sockets in the future);

Proxy settings should just work by default.

>   - access to frames. There are a few things this blocks:
>     * support for issuing JS commands to the currently focused frame (to avoid CORS errors);
>     * printing the currently focused frame;
>     * commands manipulating frame focus, listing frames, etc.;

We don't plan to expose Frames in the API unless there are use cases that justify it.

>   - access to webkit's scrollbars (uzbl has a scroll command for keybindings);

Scrolling with keys should just work too.

>   - a comparable API to WebKitWebPluginDatabase for:
>     * refreshing the plugin list;
>     * toggling specific plugins (and a property in WebKitWebPlugin for whether it is enabled);
>   - a non-async API for plugin queries;

Why do you need to do it synchronously? It might take a long time and it would block the main thread.

>   - a call to remove all web databases used by webkit;
>   - web database management (WebKitSecurityOrigin);
>   - access to webkit's spellchecking;

What do you need exactly?

>   - loading arbitrary content into a frame (webkit_web_frame_load_string);

what's the use case for this?

>   - unsetting security policies for schemes (though an API like WebKit1 with enum flags rather than an array of functions would be better anyways);

http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitSecurityManager.html

>   - access to the actual JS context of the page (webkit_web_view_get_javascript_global_context returns an isolated context);
>   - a function to get a WebKitHitTestResult at an arbitrary point on the page (uzbl has support for emulating clicks, so "last hover target" isn't necessarily useful);
>   - setting the zoom type (content or text only);

http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitSettings.html#WebKitSettings--zoom-text-only

>   - getting the current page's encoding;

http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitWebView.html#webkit-web-view-get-custom-charset

>   - get/set the total application cache maximum size;

Cache sizes are calculated automatically depending on the cache model set:

http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitWebContext.html#webkit-web-context-set-cache-model

>   - a comparable API to WebKitViewportAttributes;
>   - access to the DOM of a page/frame;

http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitWebPage.html#webkit-web-page-get-dom-document

>   - the following properties:
>     * on WebKitWebSettings:
>       - auto-resize-window,
>       - enable-universal-access-from-file-uris (ignorable if unsetting security policies is possible),
>       - enable-file-access-from-file-uris (ignorable if unsetting security policies is possible),
>       - enable-display-of-insecure-content (there is only a signal that it happened),
>       - enable-running-of-insecure-content (same as display variant),
>       - enable-scripts,
>       - enable-dom-paste,
>       - auto-shrink-images,
>       - respect-image-orientation,
>       - enable-spatial-navigation,
>       - editing-behavior,
>       - enforce-96-dpi,
>       - minimum-font-size,
>       - minimum-logical-font-size,
>       - enable-accelerated-compositing (is this always true in WebKit2?),
>       - enable-css-shaders,
>       - enable-media-stream,
>     * on WebKitWebInspector:
>       - javascript-profiling-enabled,
>       - timeline-profiling-enabled,
>     * on WebKitWebView:
>       - transparent,
>       - editable.
> 
> If I'm missing something in WebKit2 that does the same thing, I may have missed it. I've been over both sets of docs a few times and still find new things every now and then. If there is no plans to support a feature, the only ones which are 100% required for uzbl to work is access to the scrollbars, SoupSession, and page's JavaScript context.

We need use cases for those, to understand the problems.

> For further tracking of specific items, creating a new bug is probably preferable (rather than polluting this bug). However, making one bug for each from the start seems overkill if I've missed things and half are just going to be closed right away.

Yes we can use this as a meta bug and open specific bug reports blocking this.

> [1]http://uzbl.org

[Ben Boeckel]

These comments are getting quite long. Want to open bugs for things that you plan on addressing and then I'll open bugs for those which might need more discussion to convince you that they're needed/nice enough to have? After all the big stuff is done, we can deal with the more nitpicky things.

(In reply to comment #1)
> (In reply to comment #0)
> >   - the SoupSession in inaccessible which was used for:
> 
> This is simply not possible, because the network is in the web process. However, we expose in the UI process API for several network features.
> 
> >     * send events on cookie add/delete so that plugins could filter and persist cookies between instances;
> http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitCookieManager.html#WebKitCookieManager-changed

So…on every cookie change, we need to iterate over every cookie and compare it to every cookie we had before to make the same events? Since the SoupSession isn't accessible, the following APIs would be satisfactory:

  - cookie-added signal
  - cookie-removed signal
  - add_cookie API
  - remove_cookie API

This would basically get us what we are getting now with access to the SoupSession (except proxy support, see below).

> >     * a command to authenticate to a site;
> 
> HTTP authentication should just work by default.

Sure, but instead of a dialog, a user might have scripted auth to a site which means we need a signal like authentication-requested which can be blocked.

> >     * external authentication dialog handling (e.g., scripted auth support);
> 
> We plan to add API to allow handling HTTP auth request.

Sounds good.

> >     * custom proxy handling (I'm planning on using domain sockets rather than TCP sockets in the future);
> 
> Proxy settings should just work by default.

My plan was to use a per-instance proxy for uzbl to do some things which aren't easy with the WebKit API: setting/changing/removing HTTP headers, host/path based blocking (though policy-decision might be possible for this), caching, implementations of some Mozilla projects (e.g., collusion). Since a user might have 20 instances open, taking up 40 TCP ports might be overkill, so I was going to use UNIX sockets instead. This requires uzbl to be able to set a custom proxy URI and GProxy on the SoupSession to force it to talk over AF_UNIX rather than TCP (see this[1] thread). Plus, I can lock down AF_UNIX sockets to be only accessible by the user running uzbl.

> >   - access to frames. There are a few things this blocks:
> >     * support for issuing JS commands to the currently focused frame (to avoid CORS errors);
> >     * printing the currently focused frame;
> >     * commands manipulating frame focus, listing frames, etc.;
> 
> We don't plan to expose Frames in the API unless there are use cases that justify it.

Because uzbl is modal, being able to swap the current frame focus would be good for sending requests to specific frames (CORS is going to block JS we execute in the top-level frame).

> >   - access to webkit's scrollbars (uzbl has a scroll command for keybindings);
> 
> Scrolling with keys should just work too.

With hjkl? Since uzbl is modal, so space/pgup/pgdown won't work without this unless you're in 'insert' mode. Plus there are behaviors with default keys I dislike (space is better as half-page since PgDn already does full page, but whatever :) ).

> >   - a comparable API to WebKitWebPluginDatabase for:
> >     * refreshing the plugin list;
> >     * toggling specific plugins (and a property in WebKitWebPlugin for whether it is enabled);
> >   - a non-async API for plugin queries;
> 
> Why do you need to do it synchronously? It might take a long time and it would block the main thread.

There's a variable with the list of plugins; it'd be nice to populate it on demand, but I suppose I could make a command to populate these async properties in the background.

> >   - a call to remove all web databases used by webkit;
> >   - web database management (WebKitSecurityOrigin);
> >   - access to webkit's spellchecking;
> 
> What do you need exactly?

WebKit1 had the ability to teach words, get suggestions, etc.

> >   - loading arbitrary content into a frame (webkit_web_frame_load_string);
> 
> what's the use case for this?

WebKit1 had it. Without it, we can only load text/plain and text/html. I'm not married to its fate in WebKit2.

> >   - unsetting security policies for schemes (though an API like WebKit1 with enum flags rather than an array of functions would be better anyways);
> 
> http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitSecurityManager.html

*Un*setting policies :) . Those APIs only let us set them.

> >   - access to the actual JS context of the page (webkit_web_view_get_javascript_global_context returns an isolated context);

?

> >   - a function to get a WebKitHitTestResult at an arbitrary point on the page (uzbl has support for emulating clicks, so "last hover target" isn't necessarily useful);

This doesn't seem to work with WebKit2. The test you pointed me at simulates mouse moving. In any case, I'm not seeing LINK_HOVER events when doing the click emulation, so we wouldn't be able to just use "last hover target".

> >   - setting the zoom type (content or text only);
> 
> http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitSettings.html#WebKitSettings--zoom-text-only

Ah, I had named the bindings differently between WebKit1 and WebKit2. Thanks.

> >   - getting the current page's encoding;
> 
> http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitWebView.html#webkit-web-view-get-custom-charset

From the WebKit1 API, "custom-charset" could be different than "current-encoding". Is this no longer true?

> >   - get/set the total application cache maximum size;
> 
> Cache sizes are calculated automatically depending on the cache model set:
> 
> http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitWebContext.html#webkit-web-context-set-cache-model

Sounds good.

> >   - a comparable API to WebKitViewportAttributes;
> >   - access to the DOM of a page/frame;
> 
> http://www.webkitgtk.org/reference/webkit2gtk/stable/WebKitWebPage.html#webkit-web-page-get-dom-document
> 
> >   - the following properties:
> >     * on WebKitWebSettings:
> >       - auto-resize-window,

This behavior is annoying and it would be nice to be able to turn it off.

> >       - enable-universal-access-from-file-uris (ignorable if unsetting security policies is possible),
> >       - enable-file-access-from-file-uris (ignorable if unsetting security policies is possible),

If functions to unset policies on schemes is added, these are unrequired, otherwise once you lock down the file scheme, you can't unlock it.

> >       - enable-display-of-insecure-content (there is only a signal that it happened),
> >       - enable-running-of-insecure-content (same as display variant),

I suppose the policies also take care of these.

> >       - enable-scripts,

It would be nice to have really crude NoScript support.

> >       - enable-dom-paste,

Is javascript-can-access-clipboard a superset of this? If so, I can live with just that.

> >       - auto-shrink-images,
> >       - respect-image-orientation,

They exist in WebKit1 and users might want these behaviors.

> >       - enable-spatial-navigation,

I suppose this might be someone's preferred link navigation method… Personally, I just use a link labeller.

> >       - editing-behavior,

Maybe someone likes editing behavior other than "native"?

> >       - enforce-96-dpi,

This is under "hack" anyways, so this can disappear without qualms.

> >       - minimum-font-size,
> >       - minimum-logical-font-size,

WebKit2 is showing much smaller fonts than WebKit1 with the same settings; I'll have to do some investigating as to whether it's just a WebKit1 -> WebKit2 change or these variables are helping WebKit1.

> >       - enable-accelerated-compositing (is this always true in WebKit2?),
> >       - enable-css-shaders,
> >       - enable-media-stream,

I'm guessing these are always enabled in WebKit2. I imagine they were optional because they were unstable. I'm fine with living without these.

> >     * on WebKitWebInspector:
> >       - javascript-profiling-enabled,
> >       - timeline-profiling-enabled,

The timeline-profiling-enabled property was already deprecated, but it'd be nice to be able to control the inspector without a mouse (to click the buttons for these).

> >     * on WebKitWebView:
> >       - transparent,
> >       - editable.

These behaviors are really nice to have (I use editable at least).

> > If I'm missing something in WebKit2 that does the same thing, I may have missed it. I've been over both sets of docs a few times and still find new things every now and then. If there is no plans to support a feature, the only ones which are 100% required for uzbl to work is access to the scrollbars, SoupSession, and page's JavaScript context.
> 
> We need use cases for those, to understand the problems.

I don't really have hard use cases for all of them. A few are just "huh, where'd that go?" when moving over to WebKit2. Uzbl is basically a browser which tries to expose all WebKit settings to the user. In the end, it'd be nice to have all of these so that I can remove a lot of #ifndef USE_WEBKIT2 blocks from the code (or at least have #else sections for them).

[1]https://mail.gnome.org/archives/libsoup-list/2013-January/msg00001.html

[Carlos Garcia Campos]

(In reply to comment #2)
> These comments are getting quite long. Want to open bugs for things that you plan on addressing and then I'll open bugs for those which might need more discussion to convince you that they're needed/nice enough to have? After all the big stuff is done, we can deal with the more nitpicky things.

We can file bug reports for the things we plan to do and discuss the other things in the mailing list.

[...]

> I don't really have hard use cases for all of them. A few are just "huh, where'd that go?" when moving over to WebKit2. Uzbl is basically a browser which tries to expose all WebKit settings to the user. In the end, it'd be nice to have all of these so that I can remove a lot of #ifndef USE_WEBKIT2 blocks from the code (or at least have #else sections for them).

We are trying to keep the API as simple as possible, so unless there's a real need for something we will not add more API, and unfortunately things like "it was available in WebKit1" or "it would be nice to have" are not enough.

[Carlos Garcia Campos]

(In reply to comment #2)

> > >       - minimum-font-size,
> > >       - minimum-logical-font-size,
> 
> WebKit2 is showing much smaller fonts than WebKit1 with the same settings; I'll have to do some investigating as to whether it's just a WebKit1 -> WebKit2 change or these variables are helping WebKit1.

Font sizes in WebKit2 are in pixels, not points.

[Martin Robinson]

> My plan was to use a per-instance proxy for uzbl to do some things which aren't easy with the WebKit API: setting/changing/removing HTTP headers, host/path based blocking (though policy-decision might be possible for this), caching, implementations of some Mozilla projects (e.g., collusion). Since a user might have 20 instances open, taking up 40 TCP ports might be overkill, so I was going to use UNIX sockets instead. This requires uzbl to be able to set a custom proxy URI and GProxy on the SoupSession to force it to talk over AF_UNIX rather than TCP (see this[1] thread). Plus, I can lock down AF_UNIX sockets to be only accessible by the user running uzbl.

A lot of these things (perhaps all?) could be handled more efficiently via a WebExtension. The proxy approach is kind of funky.

> With hjkl? Since uzbl is modal, so space/pgup/pgdown won't work without this unless you're in 'insert' mode. Plus there are behaviors with default keys I dislike (space is better as half-page since PgDn already does full page, but whatever :) ).

This could probably be accomplished with execute_comamnd('MoveDown') or some translation of key events.
 
> > >   - a call to remove all web databases used by webkit;
> > >   - web database management (WebKitSecurityOrigin);
> > >   - access to webkit's spellchecking;
> > 
> > What do you need exactly?
> 
> WebKit1 had the ability to teach words, get suggestions, etc. 

This API was there because of limitations in the context menu. It was difficult to combine the items from WebCore with custom items. That's a lot easier now, so we don't really need the API. You can always talk directly to enchant though.

> > >       - editing-behavior,
> 
> Maybe someone likes editing behavior other than "native"?

I exposed this setting originally and regret it. Editing behavior should just follow the system convention. Note that WebKit still obeys the user's key bindings.

> 
> > >       - enforce-96-dpi,
> 
> This is under "hack" anyways, so this can disappear without qualms.

As Carlos said, font sizes are now in pixels instead of points, so this setting is now meaningless. :)

> WebKit2 is showing much smaller fonts than WebKit1 with the same settings; I'll have to do some investigating as to whether it's just a WebKit1 -> WebKit2 change or these variables are helping WebKit1.
> 
> > >       - enable-accelerated-compositing (is this always true in WebKit2?),
> > >       - enable-css-shaders,
> > >       - enable-media-stream,

Accelerated compositing is always on depending on whether your system supports it. I'm not sure if we're going to expose many settings for basic web platform features, since there isn't a clear demarcation of where we should stop.

> > >     * on WebKitWebInspector:
> > >       - javascript-profiling-enabled,
> > >       - timeline-profiling-enabled,
> 
> The timeline-profiling-enabled property was already deprecated, but it'd be nice to be able to control the inspector without a mouse (to click the buttons for these).

It's important that the web inspector is accessible and can be controlled without a mouse. Perhaps we can ensure that's possible without exposing new settings.

> 
> > >     * on WebKitWebView:
> > >       - transparent,
> > >       - editable.
> 
> These behaviors are really nice to have (I use editable at least).
> 
> > > If I'm missing something in WebKit2 that does the same thing, I may have missed it. I've been over both sets of docs a few times and still find new things every now and then. If there is no plans to support a feature, the only ones which are 100% required for uzbl to work is access to the scrollbars, SoupSession, and page's JavaScript context.

To access the page's JavaScript context and SoupSession you'll need to create a WebExtension. This is something inherent in the design of WebKit2, so I don't think we'll be able to change it. Even for the SoupSession, it's likely that we'll never *really* expose it since networking is moving to another process entirely.

Thanks for making this list. Hopefully we can work together to solve the missing features by either adding them or improving the uzbl code to the point that they are no longer necessary.

[Ben Boeckel]

(In reply to comment #5)
> > My plan was to use a per-instance proxy for uzbl to do some things which aren't easy with the WebKit API: setting/changing/removing HTTP headers, host/path
 based blocking (though policy-decision might be possible for this), caching, implementations of some Mozilla projects (e.g., collusion). Since a user might ha
ve 20 instances open, taking up 40 TCP ports might be overkill, so I was going to use UNIX sockets instead. This requires uzbl to be able to set a custom proxy URI and GProxy on the SoupSession to force it to talk over AF_UNIX rather than TCP (see this[1] thread). Plus, I can lock down AF_UNIX sockets to be only acce
ssible by the user running uzbl.
> 
> A lot of these things (perhaps all?) could be handled more efficiently via a WebExtension.

The API for WebExtensions seems a little bare which is why I skipped it before. Is there an example of an extension that does something like this.

> The proxy approach is kind of funky.

I can't say I disagree, but do you have another suggestion that would work with per-instance proxies? That aren't accessible from users other than the user running uzbl.

> > With hjkl? Since uzbl is modal, so space/pgup/pgdown won't work without this unless you're in 'insert' mode. Plus there are behaviors with default keys I dislike (space is better as half-page since PgDn already does full page, but whatever :) ).
> 
> This could probably be accomplished with execute_comamnd('MoveDown') or some translation of key events.

Currently supported with WebKit1:

  - page-relative: move relative to page size (50% for half page, -25% for up a quarter page);
  - unit-relative: "units" (50 goes down 50 "units" of whatever the scrollbar is reporting);
  - named-location: "begin" and "end" for home/end;
  - absolute-location: 20! moves to location 20 "units" on the scrollbar;
  - relative-location: 20%! moves to 20% of the total scrollbar height.

If you want to add commands for these (all of which except absolute-location have definite uses (relative-location is nice when restoring a page and named-location can probably be turned into 0%! and 100%!).

> > > >   - a call to remove all web databases used by webkit;
> > > >   - web database management (WebKitSecurityOrigin);
> > > >   - access to webkit's spellchecking;
> > > 
> > > What do you need exactly?
> > 
> > WebKit1 had the ability to teach words, get suggestions, etc. 
> 
> This API was there because of limitations in the context menu. It was difficult to combine the items from WebCore with custom items. That's a lot easier now, so we don't really need the API. You can always talk directly to enchant though.

I assume that the enchant instance the webpage uses for edit boxes is beyond the IPC call? For non-webkit-associated spellchecking, the user can just exec hunspell.

> > > >       - editing-behavior,
> > 
> > Maybe someone likes editing behavior other than "native"?
> 
> I exposed this setting originally and regret it. Editing behavior should just follow the system convention. Note that WebKit still obeys the user's key bindings.
> 
> > > >       - enforce-96-dpi,
> > 
> > This is under "hack" anyways, so this can disappear without qualms.
> 
> As Carlos said, font sizes are now in pixels instead of points, so this setting is now meaningless. :)

Makes sense.

> > > >       - enable-accelerated-compositing (is this always true in WebKit2?),
> > > >       - enable-css-shaders,
> > > >       - enable-media-stream,
> 
> Accelerated compositing is always on depending on whether your system supports it. I'm not sure if we're going to expose many settings for basic web platform features, since there isn't a clear demarcation of where we should stop.

I'm fine with that.

> > > >     * on WebKitWebInspector:
> > > >       - javascript-profiling-enabled,
> > > >       - timeline-profiling-enabled,
> > 
> > The timeline-profiling-enabled property was already deprecated, but it'd be nice to be able to control the inspector without a mouse (to click the buttons for these).
> 
> It's important that the web inspector is accessible and can be controlled without a mouse. Perhaps we can ensure that's possible without exposing new settings.

In that case, I'll see if I can come up with other things that would be nice ;) .

> To access the page's JavaScript context and SoupSession you'll need to create a WebExtension. This is something inherent in the design of WebKit2, so I don't think we'll be able to change it. Even for the SoupSession, it's likely that we'll never *really* expose it since networking is moving to another process entirely.

Sounds reasonable, the WebExtensions just need either better docs or more API to do these things it looks like.

> Thanks for making this list. Hopefully we can work together to solve the missing features by either adding them or improving the uzbl code to the point that they are no longer necessary.

The uzbl code will likely need to move over to a threaded architecture to deal with the async calls that are required (and I'm assuming any comm with our WebExtension would need to be async as well).

[Simon Pena]

Adding dependency on behalf of Brian Holt.

[Mario Sanchez Prada]

(In reply to comment #2)
> [...]
> They exist in WebKit1 and users might want these behaviors.
> 
> > >       - enable-spatial-navigation,
> 
> I suppose this might be someone's preferred link navigation method… Personally, 
> I just use a link labeller.
> 
Just for the sake of clarity, this feature would be interesting in environments where you don't have the usual input methods you happen to access to in environments such PCs or touch devices (e.g. smartphones, tablets..), such as TVs, where you use a remote controller to navigate around the web.

It's pretty much related to bug 114298, which is pending on WK2 owners to approve the cross platform changes.

[Ben Boeckel]

So to remake the list for clarity (anything missing from the original list is obsolete):

>   - the SoupSession in inaccessible which was used for:
>     * send events on cookie add/delete so that plugins could filter and persist cookies between instances;

There's WebKitCookieManager::changed, but getting a cookie structure with the signal would be nice (maybe ::added, ::removed, and ::modified all calling callback(WebKitCookieManager*, WebKitCookie*) or something)). Also the ability to manually add/remove cookies for more fine-grained policy making outside of first/third party groupings.

>   - access to frames. There are a few things this blocks:
>     * support for issuing JS commands to the currently focused frame (to avoid CORS errors);
>     * printing the currently focused frame;
>     * commands manipulating frame focus, listing frames, etc.;
>     * loading arbitrary content into a frame (webkit_web_frame_load_string);

Probably doable via JS access to the page (via a WebKitWebExtension).

>   - access to webkit's scrollbars (uzbl has a scroll command for keybindings);
>   - access to the actual JS context of the page (webkit_web_view_get_javascript_global_context returns an isolated context);

Looks like it would need a WebKitWebExtension. An expanded WebKitWebExtension API would be nice to have and more docs on how to create/install one would be great. I'll open a separate bug for WebKitWebExtension API requests.

>   - a comparable API to WebKitWebPluginDatabase for:
>     * refreshing the plugin list;
>     * toggling specific plugins (and a property in WebKitWebPlugin for whether it is enabled);
>   - a call to remove all web databases used by webkit;
>   - web database management (WebKitSecurityOrigin);
>   - unsetting security policies for schemes (though an API like WebKit1 with enum flags rather than an array of functions would be better anyways);
>   - a function to get a WebKitHitTestResult at an arbitrary point on the page (uzbl has support for emulating clicks, so "last hover target" isn't necessarily useful);
>   - a comparable API to WebKitViewportAttributes;

?

>   - getting the current page's encoding;

This is not custom-encoding (which is just a user override of the encoding HTTP header).

>   - the following properties:
>     * on WebKitWebInspector:
>       - javascript-profiling-enabled,
>       - timeline-profiling-enabled,

More programmatic access to the Inspector is needed; I'll try to get a concrete list and open a bug for it since it's distinct enough.

>     * on WebKitWebSettings:
>       - enable-spatial-navigation,

Bug #114298.

>       - auto-resize-window,
>       - enable-scripts,
>       - enable-dom-paste,
>       - auto-shrink-images,
>       - respect-image-orientation,
>       - minimum-font-size,
>       - minimum-logical-font-size,
>     * on WebKitWebView:
>       - transparent,
>       - editable.

?

[Ben Boeckel]

>   - unsetting security policies for schemes (though an API like WebKit1 with enum flags rather than an array of functions would be better anyways);

Looking at WebCore itself, there isn't an API to do this[1]. I've filed Bug #120190 for this.

I do see, however, some missing security policies from the GTK bindings. Are there specific reasons each shouldn't be allowed?

  - setDomainRelaxationForbiddenForURLScheme (no docs; not sure exactly what it does)
  - canDisplayOnlyIfCanRequest (seems useful as another adblock mechanism?)
  - registerURLSchemeAsNotAllowingJavascriptURLs (block JS for non-secure schemes?)
  - registerURLSchemeAsAllowingLocalStorageAccessInPrivateBrowsing (nice for things like uzbl: protocol?)
  - registerURLSchemeAsAllowingDatabaseAccessInPrivateBrowsing (same as ...LocalStorage... (WebKit symbols sure are wordy...))
  - registerURLSchemeAsBypassingContentSecurityPolicy (I wonder if uzbl could inject its JS under the uzbl-internal: protocol and then set this to get access to the page without reciprocal action)
  - shouldCacheResponsesFromURLSchemeIndefinitely (might be useful for bandwidth-constrained systems?)

[1]The WebKit1 bindings in webkit_set_security_policy_for_uri_scheme() are also faked and don't unset anything, only set them :/ .

[Michael Catanzaro]

Hm, there's no good Bugzilla resolution to use for this bug... INVALID is not quite right as you have a bunch of valid requests, but this huge list in one bug is not really actionable. I encourage you to file individual bugs for individual APIs.