Bug 122952 – [GTK][WPE] Support NTLM authentication

• Component: WebKitGTK
• Status: RESOLVED FIXED

[Brian Holt]

According to libsoup API docs, NTLM authentication is not enabled by default and the if the user wants it, the feature should be added to the session explicitly. We are not doing that currently in WebKit, which means that NTLM authentication is not supported.

From Dan Winship:
    There are some arguments against enabling it by default; if you have the client-side samba stuff installed, and are logged into a Windows domain, then NTLM authentication can happen completely transparently (ie, no "authenticate" signal, no password dialog) using the cached credentials, and there are attacks against intranets that you could make using that functionality if you could hijack someone's http connection... so it's best to only have it get used when the app is explicitly expecting it to be used (as in evolution).

Instead we should expose an API in WebKit like

WEBKIT_API void
webkit_web_context_set_ntlm_authentication_enabled(WebKitWebContext *context,
                                                   gboolean          enabled);

that sends a message to the WebProcess (or NetworkProcess), which in turn will add the feature to the soup session using

  soup_session_add_feature_by_type (session, SOUP_TYPE_NTLM_AUTH);

[Denis Nomiyama (dnomi)]

I'll be happy to work on this.

[activey]

Any progress? :)

[activey]

Pretty pleeeaseee

[Denis Nomiyama (dnomi)]

Sorry for the late reply. I haven't got much time to work on this feature. Any volunteers?

[Paul van Tilburg]

Some customers of our WebKitGTK+-based application use NTLM authentication.
After my just-finished port to WebKit2GTK+, the loss of NTLM-support is indeed one of the little things that shows up as a regression.

I'd be willing to create a patch but I'm not familiar with the codebase at all!
Is there an example of a similar call?

[Carlos Garcia Campos]

Created attachment 318247 [details]
Patch

Note that this is untested, I have no idea how to test it, or whether we can tests it in our unit tests at all.

[Build Bot]

Thanks for the patch. If this patch contains new public API please make sure it follows the guidelines for new WebKit2 GTK+ API. See http://trac.webkit.org/wiki/WebKitGTK/AddingNewWebKit2API

[Michael Catanzaro]

Comment on attachment 318247 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=318247&action=review

Hey Paul, can you try this patch please and report back as to whether it works or not? r=me if it works.

> Source/WebCore/platform/network/soup/SoupNetworkSession.cpp:338
> +    if (gInitialNTLMAuthenticationEnabled == enabled)
> +        return;

This is wrong. Did you copy this from somewhere (like ignore TLS errors)? It should be easy to see that this breaks if NTLM was initially enabled and later disabled, or vice-versa.

> Source/WebKit/UIProcess/API/glib/WebKitWebContext.cpp:1554
> + * is disabled, you need to call webkit_web_context_set_ntlm_authentication_enabled()

It's a run-on sentence. The two best options to fix it:

"disabled, and you"

"disabled. You"

[Carlos Garcia Campos]

Comment on attachment 318247 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=318247&action=review

>> Source/WebCore/platform/network/soup/SoupNetworkSession.cpp:338
>> +        return;
> 
> This is wrong. Did you copy this from somewhere (like ignore TLS errors)? It should be easy to see that this breaks if NTLM was initially enabled and later disabled, or vice-versa.

No, I didn't copy it, but you are right, this is wrong. Actually we always call first the Initial and then this, so it's always the same :-P

[Carlos Garcia Campos]

Created attachment 318267 [details]
Updated patch

[Michael Catanzaro]

Comment on attachment 318267 [details]
Updated patch

OK, r=me if Paul or someone else reports that it actually works.

[Paul van Tilburg]

Comment on attachment 318267 [details]
Updated patch

I've used the API call and tried opening a test page with NTLM auth. It works!

(I hope I am doing this review thing correctly.)

[Carlos Garcia Campos]

Committed r220854: <http://trac.webkit.org/changeset/220854>

[Carlos Garcia Campos]

I've been told that we don't really need this. We already support gssapi, and that supports ntlm if the right package is installed. Paul, could you confirm it works for you by installing gss-ntlmssp and without the patch? What libsoup version are you using, btw?

[Tomas Popela]

(In reply to Carlos Garcia Campos from comment #14)
> confirm it works for you by installing gss-ntlmssp and without the patch?
> What libsoup version are you using, btw?

Actually on Fedora it's gssntlmssp and the libsoup version should be 2.54+.

[WebKit Commit Bot]

Re-opened since this is blocked by bug 175716

[Paul van Tilburg]

Sorry, I got sidetracked and also had to get libsoup 2.56.0 backported to Ubuntu Xenial first to be able to test this.

I installed gss-ntlmssp and installed libsoup2.4 2.56.0, but nothing happens for me, really.  When opening a page that sends WWW-Authenticate: NTLM, WebKitGTK basically doesn't react at all (without the patch) and the user gets a 401.

If I look at commit 6a8826f0d25459c943719b88f22619f22b81c547 in libsoup, I see it just provides an implementation for WWW-Authenticate: Negotiate next to the already existing WWW-Authenticate: NTLM, where the latter is the one I'm trying to support.  It is not clear to me from documentation how supporting one is also covering supporting the other.

However, it would be great if we could drop the patch.  Ubuntu often updates WebKitGTK (which is a good thing), but it is a pain to keep doing the non-trivial rebuild for this patch all the time ;)

[Paul van Tilburg]

Years have passed and I can confirm that in Ubuntu 18.04 LTS (Bionic) works out of the box using the normal authentication callbacks, so this bug report can be closed.

[Michael Catanzaro]

(In reply to Brian Holt from comment #0)
> From Dan Winship:
>     There are some arguments against enabling it by default; if you have the
> client-side samba stuff installed, and are logged into a Windows domain,
> then NTLM authentication can happen completely transparently (ie, no
> "authenticate" signal, no password dialog) using the cached credentials, and
> there are attacks against intranets that you could make using that
> functionality if you could hijack someone's http connection... so it's best
> to only have it get used when the app is explicitly expecting it to be used
> (as in evolution).
> 
> Instead we should expose an API in WebKit like
> 
> WEBKIT_API void
> webkit_web_context_set_ntlm_authentication_enabled(WebKitWebContext *context,
>                                                    gboolean         
> enabled);
> 
> that sends a message to the WebProcess (or NetworkProcess), which in turn
> will add the feature to the soup session using
> 
>   soup_session_add_feature_by_type (session, SOUP_TYPE_NTLM_AUTH);

Firefox enables it by default. And if it's not enabled, you cannot access websites that are gated by NTLM auth. For web compat, I think we have to match other browsers.

(In reply to Carlos Garcia Campos from comment #14)
> I've been told that we don't really need this. We already support gssapi,
> and that supports ntlm if the right package is installed. Paul, could you
> confirm it works for you by installing gss-ntlmssp and without the patch?
> What libsoup version are you using, btw?

In Fedora the package is gssntlmssp. I confirmed that installing the package is not enough to make NTLM work on the test page http://ntlm.herokuapp.com/. I use gssapi every day for kerberos auth, and that works fine. So only NTML is not working. Reopening.

(In reply to Paul van Tilburg from comment #18)
> Years have passed and I can confirm that in Ubuntu 18.04 LTS (Bionic) works
> out of the box using the normal authentication callbacks, so this bug report
> can be closed.

I don't know how to explain this. Maybe something broke between then and now.

[Michael Catanzaro]

*** Bug 215920 has been marked as a duplicate of this bug. ***

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #19)
> Firefox enables it by default. And if it's not enabled, you cannot access
> websites that are gated by NTLM auth. For web compat, I think we have to
> match other browsers.

Also, reading the Wikipedia page, it seems clear that NTLM is insecure no matter what. It also looks like it really ought to work via gssapi. I wonder why it doesn't.

[Michael Catanzaro]

I tried:

diff --git a/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp b/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
index 498d1ac57277..9f8b71ac1a93 100644
--- a/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
+++ b/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
@@ -132,6 +132,7 @@ SoupNetworkSession::SoupNetworkSession(PAL::SessionID sessionID)
 #if SOUP_CHECK_VERSION(2, 67, 90)
         SOUP_SESSION_ADD_FEATURE_BY_TYPE, SOUP_TYPE_WEBSOCKET_EXTENSION_MANAGER,
 #endif
+        SOUP_SESSION_ADD_FEATURE_BY_TYPE, SOUP_TYPE_AUTH_NTLM,
         nullptr);
 
     if (!initialAcceptLanguages().isNull())

Since if we don't need API to enable/disable NTLM, all of the complexity of Carlos's original patch goes away. Unfortunately, the test page fails. In Firefox, I see a username/password prompt. In Epiphany, with this change, I see the text: "You are not authorized to see this page." Without this change, I just see a blank page. So looks like it needs a lot more investigation to get working.

[Carlos Garcia Campos]

(In reply to Michael Catanzaro from comment #22)
> I tried:
> 
> diff --git a/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
> b/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
> index 498d1ac57277..9f8b71ac1a93 100644
> --- a/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
> +++ b/Source/WebCore/platform/network/soup/SoupNetworkSession.cpp
> @@ -132,6 +132,7 @@ SoupNetworkSession::SoupNetworkSession(PAL::SessionID
> sessionID)
>  #if SOUP_CHECK_VERSION(2, 67, 90)
>          SOUP_SESSION_ADD_FEATURE_BY_TYPE,
> SOUP_TYPE_WEBSOCKET_EXTENSION_MANAGER,
>  #endif
> +        SOUP_SESSION_ADD_FEATURE_BY_TYPE, SOUP_TYPE_AUTH_NTLM,
>          nullptr);
>  
>      if (!initialAcceptLanguages().isNull())
> 
> Since if we don't need API to enable/disable NTLM, all of the complexity of
> Carlos's original patch goes away. Unfortunately, the test page fails. In
> Firefox, I see a username/password prompt. In Epiphany, with this change, I
> see the text: "You are not authorized to see this page." Without this
> change, I just see a blank page. So looks like it needs a lot more
> investigation to get working.

It works for me with MiniBrowser. I don't know what have changed (if any), but this test site uses NTLM in WWW-Authenticate header, and that's only handled by SoupAuthNTLM, so the feature needs to be added.

[Michael Catanzaro]

(In reply to Carlos Garcia Campos from comment #23)
> It works for me with MiniBrowser.

Hm, yes, works in MiniBrowser for me too. Well, this is a one-liner to fix in WebKit, then, unless we want all the plumbing to keep in disabled by default. I'll ask Dan Winship to comment. His opinion was previously that we should not enable NTLM by default, but it's clear that Firefox does, so ?????.

Anyway, if I remove Epiphany's authenticate_cb (ephy-web-view.c), then it works in Ephy too. The difference is that MiniBrowser doesn't implement the authenticate signal. Epiphany returns TRUE to handle the authentication event, then does a password manager lookup. If it has a stored username/password, then it uses that; otherwise, it calls webkit_authentication_request_authenticate() with an empty credential:

webkit_credential_new (" ", "", WEBKIT_CREDENTIAL_PERSISTENCE_NONE);

That normally triggers a retry, so the user gets the default auth prompt. But with NTLM, it just results in an error. I guess we are actually authenticating to the peer with empty username/password each time we display a dialog for which we don't have username/password saved already. So, that's not great. Ideally we would have some function to allow us to display WebKit's default authentication prompt after the authenticate callback without actually having to send credentials to the server.

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #24)
> Hm, yes, works in MiniBrowser for me too. Well, this is a one-liner to fix
> in WebKit, then, unless we want all the plumbing to keep in disabled by
> default. I'll ask Dan Winship to comment. His opinion was previously that we
> should not enable NTLM by default, but it's clear that Firefox does, so
> ?????.

Reading comment #0, I see the concern is that a MITM attacker can force the browser to give up cached credentials if authentication occurs via http:// rather than https://:

(In reply to Brian Holt from comment #0)
> From Dan Winship:
>     There are some arguments against enabling it by default; if you have the
> client-side samba stuff installed, and are logged into a Windows domain,
> then NTLM authentication can happen completely transparently (ie, no
> "authenticate" signal, no password dialog) using the cached credentials, and
> there are attacks against intranets that you could make using that
> functionality if you could hijack someone's http connection... so it's best
> to only have it get used when the app is explicitly expecting it to be used
> (as in evolution).

Well, that applies to other authentication schemes as well, since, as I just described, Ephy caches credentials for all schemes. Epiphany segregates this by protocol, so credentials for an https:// page will never be used for an http:// page. Regardless of whether or not Samba does that, I understand NTLM is considered generally insecure, so hopefully organizations that care about security won't use it. And certainly we don't have to be more secure than Firefox. So I guess enabling it by default is OK.

[Carlos Garcia Campos]

(In reply to Michael Catanzaro from comment #24)
> (In reply to Carlos Garcia Campos from comment #23)
> > It works for me with MiniBrowser.
> 
> Hm, yes, works in MiniBrowser for me too. Well, this is a one-liner to fix
> in WebKit, then, unless we want all the plumbing to keep in disabled by
> default. I'll ask Dan Winship to comment. His opinion was previously that we
> should not enable NTLM by default, but it's clear that Firefox does, so
> ?????.
> 
> Anyway, if I remove Epiphany's authenticate_cb (ephy-web-view.c), then it
> works in Ephy too. The difference is that MiniBrowser doesn't implement the
> authenticate signal. Epiphany returns TRUE to handle the authentication
> event, then does a password manager lookup. If it has a stored
> username/password, then it uses that; otherwise, it calls
> webkit_authentication_request_authenticate() with an empty credential:
> 
> webkit_credential_new (" ", "", WEBKIT_CREDENTIAL_PERSISTENCE_NONE);
> 
> That normally triggers a retry, so the user gets the default auth prompt.
> But with NTLM, it just results in an error. I guess we are actually
> authenticating to the peer with empty username/password each time we display
> a dialog for which we don't have username/password saved already.

That's probably a different behavior in libsoup rather than WebKit. Maybe we relied on specific auth type behavior for the api, I'll investigate.

> So, that's
> not great. Ideally we would have some function to allow us to display
> WebKit's default authentication prompt after the authenticate callback
> without actually having to send credentials to the server.

I agree.

[Carlos Garcia Campos]

Do other browsers allow to disable it even if it's enabled by default?

[Michael Catanzaro]

I don't think Firefox allows disabling it.

[Dan Winship]

I have no deep insight about this. If people who actually work on WebKitGTK think it makes sense for it to be enabled by default then it should be enabled by default there I guess.

[Carlos Garcia Campos]

Created attachment 408421 [details]
Patch

[Carlos Garcia Campos]

(In reply to Carlos Garcia Campos from comment #26)
> (In reply to Michael Catanzaro from comment #24)
> > (In reply to Carlos Garcia Campos from comment #23)
> > > It works for me with MiniBrowser.
> > 
> > Hm, yes, works in MiniBrowser for me too. Well, this is a one-liner to fix
> > in WebKit, then, unless we want all the plumbing to keep in disabled by
> > default. I'll ask Dan Winship to comment. His opinion was previously that we
> > should not enable NTLM by default, but it's clear that Firefox does, so
> > ?????.
> > 
> > Anyway, if I remove Epiphany's authenticate_cb (ephy-web-view.c), then it
> > works in Ephy too. The difference is that MiniBrowser doesn't implement the
> > authenticate signal. Epiphany returns TRUE to handle the authentication
> > event, then does a password manager lookup. If it has a stored
> > username/password, then it uses that; otherwise, it calls
> > webkit_authentication_request_authenticate() with an empty credential:
> > 
> > webkit_credential_new (" ", "", WEBKIT_CREDENTIAL_PERSISTENCE_NONE);
> > 
> > That normally triggers a retry, so the user gets the default auth prompt.
> > But with NTLM, it just results in an error. I guess we are actually
> > authenticating to the peer with empty username/password each time we display
> > a dialog for which we don't have username/password saved already.
> 
> That's probably a different behavior in libsoup rather than WebKit. Maybe we
> relied on specific auth type behavior for the api, I'll investigate.

I have debugged this and I think it's a problem specific to the test case, after a wrong auth the server responds with a 401, but without including the WWW-Authenticate header, so we don't know how to authenticate again, see:

> GET / HTTP/1.1
> Soup-Debug-Timestamp: 1599738858
> Soup-Debug: SoupSession 1 (0x55a42834e100), SoupMessage 1 (0x55a4283748c0), SoupSocket 1 (0x55a42869e0b0)
> Host: ntlm.herokuapp.com
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Upgrade-Insecure-Requests: 1
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Safari/605.1.15
> Accept-Encoding: gzip, deflate
> Accept-Language: es-ES
> Connection: Keep-Alive

< HTTP/1.1 401 Unauthorized
< Soup-Debug-Timestamp: 1599738860
< Soup-Debug: SoupMessage 1 (0x55a4283748c0)
< Connection: keep-alive
< Www-Authenticate: NTLM
< Server: WEBrick/1.3.1 (Ruby/2.0.0/2014-09-19)
< Date: Thu, 10 Sep 2020 11:54:18 GMT
< Content-Length: 0
< Via: 1.1 vegur
< 

This is the initial request, we get Www-Authenticate: NTLM, so we create a new request with the Authorization.

> GET / HTTP/1.1
> Soup-Debug-Timestamp: 1599740315
> Soup-Debug: SoupSession 1 (0x55e751570100), SoupMessage 1 (0x55e7515968c0), SoupSocket 1 (0x55e7518c18b0), restarted
> Host: ntlm.herokuapp.com
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Upgrade-Insecure-Requests: 1
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Safari/605.1.15
> Accept-Encoding: gzip, deflate
> Accept-Language: es-ES
> Connection: Keep-Alive
> Authorization: NTLM TlRMTVNTUAABAAAABYIIAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAwAAAA

< HTTP/1.1 401 Unauthorized
< Soup-Debug-Timestamp: 1599740316
< Soup-Debug: SoupMessage 1 (0x55e7515968c0)
< Connection: keep-alive
< Www-Authenticate: NTLM TlRMTVNTUAACAAAAAAAAACgAAAABAAAAAAAAAAAAAAA=
< Server: WEBrick/1.3.1 (Ruby/2.0.0/2014-09-19)
< Date: Thu, 10 Sep 2020 12:18:36 GMT
< Content-Length: 0
< Via: 1.1 vegur
< 

We don't emit the authenticate signal in this case because soup_auth_is_authenticated() is true (password_state != NONE and != REJECTED).

Now we send the actual Authorization with the credentials (I don't know much about NTLM, but I guess it works this way):

> GET / HTTP/1.1
> Soup-Debug-Timestamp: 1599740316
> Soup-Debug: SoupSession 1 (0x55e751570100), SoupMessage 1 (0x55e7515968c0), SoupSocket 1 (0x55e7518c18b0), restarted
> Host: ntlm.herokuapp.com
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Upgrade-Insecure-Requests: 1
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Safari/605.1.15
> Accept-Encoding: gzip, deflate
> Accept-Language: es-ES
> Connection: Keep-Alive
> Authorization: NTLM TlRMTVNTUAADAAAAGAAYAFAAAAAYABgAaAAAAAAAAABAAAAAAgACAEAAAAAOAA4AQgAAAAAAAAAAAAAAAYIAACAAVQBOAEsATgBPAFcATgApMxDEA8jwjLwJGQYSjqcaV+mht5VAw3RG6ixFGUNdZlYbPMMRYwTD6Tg6rz4ENxQ=

< HTTP/1.1 401 Unauthorized
< Soup-Debug-Timestamp: 1599740316
< Soup-Debug: SoupMessage 1 (0x55e7515968c0)
< Connection: keep-alive
< Server: WEBrick/1.3.1 (Ruby/2.0.0/2014-09-19)
< Date: Thu, 10 Sep 2020 12:18:36 GMT
< Content-Length: 39
< Via: 1.1 vegur
< 

And we get another 401, but this time with no Authenticate header, so we don't emit the authenticate signal again.

> > So, that's
> > not great. Ideally we would have some function to allow us to display
> > WebKit's default authentication prompt after the authenticate callback
> > without actually having to send credentials to the server.
> 
> I agree.

[Carlos Garcia Campos]

Maybe that server is protecting against brute-force attacks by not allowing retries.

[Carlos Garcia Campos]

Committed r266819: <https://trac.webkit.org/changeset/266819>

(In reply to Carlos Garcia Campos from comment #33)
> Committed r266819: <https://trac.webkit.org/changeset/266819>

Hello Carlos,thanks for the fix.In which version of webkitgtk we can expect this fix?

[Carlos Garcia Campos]

WebKitGTK 2.32.0 that will be released tomorrow.

Tha(In reply to Carlos Garcia Campos from comment #35)
> WebKitGTK 2.32.0 that will be released tomorrow.

Thanks,will test it

[Carlos Garcia Campos]

(In reply to Carlos Garcia Campos from comment #35)
> WebKitGTK 2.32.0 that will be released tomorrow.

I meant 2.30.0