Bug 163166 - [GStreamer] Certificate database missing when accessing HTTPS via WebKitWebSourceGStreamer
Summary: [GStreamer] Certificate database missing when accessing HTTPS via WebKitWebSo...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: Other
Hardware: PC Linux
: P3 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-10-08 10:00 PDT by Jana Saout
Modified: 2016-10-10 06:25 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jana Saout 2016-10-08 10:00:40 PDT
Using WebkitGtk 2.14.0 (I don't know if this issue is relater to the GStreamer code, WK2 or SOUP interface, but it happens on my WebkitGtk branch, so I picke [GTK]):

A video tag linking to a .m3u8 file (e.g. this one)

https://video.twimg.com/ext_tw_video/783084968578723841/pu/pl/b-QJrB5LSssa0f07.m3u8

will make GStreamer choose the "HLS" plugin, which will try to create a HTTP source to read that particular file.

The gstreamer interface will attach this to WebKitWebSourceGStreamer, which will start the request using libsoup.

However, this always fails with "unacceptable TLS certificate" for me. I did some chasing:

The TLS connection in use doesn't have any certificate database set (i.e. the GObject property "database" of GTlsConnectionGnutls is NULL).

Some debugging output pointed out that for ALL other HTTP connections, there is a non-NULL database set.

Digging further yielded that this HTTP connection is made from the WebKitWebProcess task whereas all other requests are madefromthe WebkitNetworkProcess task.

This is the point where my knowledge ends because I don't know what was supposed to happen here. In case all HTTP traffic is supposed to to through the network process, you have the issue there. If not, the issue might be that the web process doesn't have access to the certificate database.
Comment 1 Michael Catanzaro 2016-10-08 12:00:16 PDT
(In reply to comment #0) 
> Digging further yielded that this HTTP connection is made from the
> WebKitWebProcess task whereas all other requests are madefromthe
> WebkitNetworkProcess task.

It will have to be fixed. If the GStreamer backend is doing network connections from the web process, how could it possibly respect our existing API for TLS errors? allow_specific_https_certificate_for_host, load-failed-with-tls-errors, etc.

(Also, we still plan to block the web process from accessing the network.)
Comment 2 Jana Saout 2016-10-09 15:52:32 PDT
Ok, after a bit of digging around I found out why all other cases are working and only the HLS plugin is having this issue. This is because only with that plugin I see network from the web process.

This is caused by this commit: http://trac.webkit.org/changeset/202615
(see bug #144040)

The new code path handling the "not main thread" case is doing this when the callback comes in from state READY to PAUSED.

This happens in webKitWebSrcStart:

    if (!priv->player || !priv->createdInMainThread) {
        priv->client = std::make_unique<ResourceHandleStreamingClient>(src, WTFMove(request));

I haven't figured out yet what this code is doing, it looks like it's setting up a new thread with its own main loop to handle the request, apparently to avoid some deadlocks.

The code path for the "from the main thread" below case does things a bit differently:

        priv->resource = priv->loader->requestResource(request, loadOptions);
        if (priv->resource) {
            priv->resource->setClient(std::make_unique<CachedResourceStreamingClient>(protector.get()));
            GST_DEBUG_OBJECT(protector.get(), "Started request");

... which goes via priv->loader wich is somehow a loader created elsewhere which at some point uses the platform strategies loader, which goes through the network process.

Somehow the special case needs to be fixed too.
Comment 3 Carlos Garcia Campos 2016-10-10 00:22:44 PDT
This is not a regression of r202615, In r202615 we only moved loads that are started in a secondary thread to another secondary thread to avoid deadlocks because of the GST locks. But before, those loas were still done in the web process using a ResourceHandle, but in the main thread. Ideally, we would not be doing those loads at all, but since our source element is public (and can't be made private with current GST APIs), we end up doing those loads. Because of that we have this second path were we use a ResourceHandle to do the loads like the souphttpsrc element would do. HLS code in GST uses a source element to download the manifest and fragments, but when using the souphttpsrc element, it also rejects connections to invalid certificates, because souphttpsrc uses ssl strict by default. So, I don't think we should be using the TLS exceptions added via API for these loads, because they don't belong to WebKit, but to GST. What I think that could have happened is that we removed the network setup from the web process in r203774, causing SoupSession object created in the web process to not have the SOUP_SESSION_SSL_USE_SYSTEM_CA_FILE property set to TRUE. Fortunately that should be fixed in trunk now by r206772
Comment 4 Jana Saout 2016-10-10 01:49:56 PDT
Oh, I see. The fallthrough case was there before already.

The case where "priv->player" is null.

Player is set from "notify::source" on the pipeline (via sourceChangedCallback).

So if the code can associate a GStreamer source to the player and this source happens to be a WEBKIT_WEB_SRC, priv->player is set (via webKitWebSrcSetMediaPlayer).

The HLS plugin can also create more WebKitWebSrc'es. Some of them fail to be associated with the player and take that code path.

I haven't tested r206772 yet, but it still doesn't sound right to me that some connections are made via the network process and some others aren't.
Comment 5 Carlos Garcia Campos 2016-10-10 01:59:14 PDT
I insist, those loads don't belong to WebKit, they are done only because GST is using our source element, which is not meant to be used for this. But since we can't avoid this from happening, we just load those requests like if they were done by souphttpsrc element. They shouldn't be sent to the network process, and they shouldn't be handled by our caches at all.
Comment 6 Jana Saout 2016-10-10 02:30:04 PDT
I see your point. The video source just makes addtional connections which just happen to be HTTP. I'm not going to be the judge whether those connections should be associated with the video player or not (I just find the situation a bit weird). :-)

Also, I just wanted to let you know that the video now plays with the commit you mentioned before (r206772).
Comment 7 Michael Catanzaro 2016-10-10 05:46:42 PDT
(In reply to comment #3)
> Fortunately that should be fixed in trunk now by r206772

So how is it that SoupSession is used by GStreamer? Are GStreamer's loads going to go through our SoupNetworkSession goo or not? If they do, as I guess, then surely our API like allow_specific_https_certificate_for_host already works, and we have no further problem. I suppose that's the case if it's using WebKit's SoupSession. (But if not, then we have a security problem; it's not safe to turn off ssl-strict on GStreamer if it's not using WebKit's custom TLS error checking.)

I'm disappointed that SoupNetworkSession exists at all in the web process.