Bug 204736 – [GTK] Allows visiting webpages that use HSTS despite certificate verification failure?

• Component: WebKitGTK
• Status: NEW

[Michael Catanzaro]

Created attachment 384587 [details]
screenshot

See attached screenshot, I can visit this website that uses HSTS even though certificate verification has failed by using the Accept Risk button. This violates the HSTS spec as the user agent should not allow ignoring the verification failure.

It cannot be solved at the Epiphany level because Epiphany doesn't know HSTS is in use. WebKit fires WebKitWebView::load-failed-with-tls-errors and provides the GTlsCertificate and GTlsCertificateFlags. That's it. So WebKit should either add some sort of API to allow implementing the spec properly, or otherwise return a different error here instead, e.g. a network error rather than load-failed-with-tls-errors. (A new API would allow for nicer UI than a network error.)

[Carlos Garcia Campos]

hsts-enforced signal is not emitted for that url. Both ff and chromium allow to accept the risk and continue.

[Michael Catanzaro]

Ah OK, it's not enforced for subdomains? I understand. Should have checked with Firefox or Chrome first.

So even though it's a bad example, point remains: Ephy has no code to handle HSTS error differently than a normal certificate verification failure. If WebKit fires load-failed-with-tls-errors, then Ephy is guaranteed to do the wrong thing. WebKit would have to use normal load-failed and present a network error for this to work, which might be an OK solution, but still a bit odd given that there were actually TLS errors. So what actually happens? And do we have any tests?

[Carlos Garcia Campos]

I would need a way to reproduce it, libsoup is cancelling the message in case of tls errors according to the code, see https://gitlab.gnome.org/GNOME/libsoup/blob/master/libsoup/soup-hsts-enforcer.c#L497

[Michael Catanzaro]

To see what happens, we would need to set up a test domain that supports HTTPS but does not redirect to it by default, and which uses an invalid certificate. I thought the badssl.com example would suffice, but as you discovered, I was wrong.

(In reply to Carlos Garcia Campos from comment #3)
> I would need a way to reproduce it, libsoup is cancelling the message in
> case of tls errors according to the code, see
> https://gitlab.gnome.org/GNOME/libsoup/blob/master/libsoup/soup-hsts-
> enforcer.c#L497

Well heck, I wonder: could this possibly be related to bug #203620? Would be a mighty coincidence if not....

[Carlos Garcia Campos]

(In reply to Michael Catanzaro from comment #4)
> To see what happens, we would need to set up a test domain that supports
> HTTPS but does not redirect to it by default, and which uses an invalid
> certificate. I thought the badssl.com example would suffice, but as you
> discovered, I was wrong.
> 
> (In reply to Carlos Garcia Campos from comment #3)
> > I would need a way to reproduce it, libsoup is cancelling the message in
> > case of tls errors according to the code, see
> > https://gitlab.gnome.org/GNOME/libsoup/blob/master/libsoup/soup-hsts-
> > enforcer.c#L497
> 
> Well heck, I wonder: could this possibly be related to bug #203620? Would be
> a mighty coincidence if not....

I don't think so, we cancel the message before, when hsts-enforced signal is emitted, so on message starting it has already been cancelled.

[Michael Catanzaro]

So, what does WebKit see? What happens after the message is cancelled?

[Claudio Saavedra]

> Both ff and chromium allow to accept the risk and continue.

I tested here in Chrome and it doesn't allow you. I think we shouldn't allow either.

I wonder if you could use WebKitWebsiteDataManager to fetch the HSTS data for the site in question. If there is HSTS data you probably want to disallow continuing. This implies checking for every site that fails to load with a SSL error, I guess, so it might not be ideal.

[Claudio Saavedra]

(In reply to Claudio Saavedra from comment #7)
> > Both ff and chromium allow to accept the risk and continue.
> 
> I tested here in Chrome and it doesn't allow you. I think we shouldn't allow
> either.
> 
> I wonder if you could use WebKitWebsiteDataManager to fetch the HSTS data
> for the site in question. If there is HSTS data you probably want to
> disallow continuing. This implies checking for every site that fails to load
> with a SSL error, I guess, so it might not be ideal.

Also because: no matter the reason of the SSL failure, you should forbid continuing if there is HSTS data stored in the browser. So it doesn't matter if it's HSTS that caused the load to be cancelled.

[Michael Catanzaro]

(In reply to Claudio Saavedra from comment #7) 
> I wonder if you could use WebKitWebsiteDataManager to fetch the HSTS data
> for the site in question. If there is HSTS data you probably want to
> disallow continuing. This implies checking for every site that fails to load
> with a SSL error, I guess, so it might not be ideal.

Hm, maybe a good idea... that might even be the ideal solution.

Let's do the check at the WebKit level, though. Can't expect dozens of different applications to do such a check.

(In reply to Carlos Garcia Campos from comment #1)
> hsts-enforced signal is not emitted for that url. Both ff and chromium allow
> to accept the risk and continue.

Huh. So Claudio tested Chrome and found you can't continue. I just tested Firefox, and was also unable to continue. Could you have typed the URL wrong? It is: http://suddomain.preloaded-hsts.badssl.com/

I checked Epiphany again just now and found the continue button is actually broken. It tries to *download* the page as a resource and fails with "Error downloading: Misdirected Request" which sounds like an artifact of using a redirect to switch from http:// -> https://. I'm not sure if I tried this before, or perhaps I just saw the button and assumed it worked.

[Claudio Saavedra]

(In reply to Michael Catanzaro from comment #9)
> (In reply to Claudio Saavedra from comment #7) 
> > I wonder if you could use WebKitWebsiteDataManager to fetch the HSTS data
> > for the site in question. If there is HSTS data you probably want to
> > disallow continuing. This implies checking for every site that fails to load
> > with a SSL error, I guess, so it might not be ideal.
> 
> Hm, maybe a good idea... that might even be the ideal solution.
> 
> Let's do the check at the WebKit level, though. Can't expect dozens of
> different applications to do such a check.

Sure, makes sense. What would the API look like though?
 
> (In reply to Carlos Garcia Campos from comment #1)
> > hsts-enforced signal is not emitted for that url. Both ff and chromium allow
> > to accept the risk and continue.
> 
> Huh. So Claudio tested Chrome and found you can't continue. I just tested
> Firefox, and was also unable to continue. Could you have typed the URL
> wrong? It is: http://suddomain.preloaded-hsts.badssl.com/
> 
> I checked Epiphany again just now and found the continue button is actually
> broken. It tries to *download* the page as a resource and fails with "Error
> downloading: Misdirected Request" which sounds like an artifact of using a
> redirect to switch from http:// -> https://. I'm not sure if I tried this
> before, or perhaps I just saw the button and assumed it worked.

Here the button was not doing anything. Then I restarted ephy and now the page is actually loaded as if there was no HSTS in place at all. Not sure why this inconsistent behavior, but in this bug I would focus on the API for the above.

[Carlos Garcia Campos]

(In reply to Claudio Saavedra from comment #10)
> (In reply to Michael Catanzaro from comment #9)
> > (In reply to Claudio Saavedra from comment #7) 
> > > I wonder if you could use WebKitWebsiteDataManager to fetch the HSTS data
> > > for the site in question. If there is HSTS data you probably want to
> > > disallow continuing. This implies checking for every site that fails to load
> > > with a SSL error, I guess, so it might not be ideal.
> > 
> > Hm, maybe a good idea... that might even be the ideal solution.
> > 
> > Let's do the check at the WebKit level, though. Can't expect dozens of
> > different applications to do such a check.
> 
> Sure, makes sense. What would the API look like though?
>  
> > (In reply to Carlos Garcia Campos from comment #1)
> > > hsts-enforced signal is not emitted for that url. Both ff and chromium allow
> > > to accept the risk and continue.
> > 
> > Huh. So Claudio tested Chrome and found you can't continue. I just tested
> > Firefox, and was also unable to continue. Could you have typed the URL
> > wrong? It is: http://suddomain.preloaded-hsts.badssl.com/
> > 
> > I checked Epiphany again just now and found the continue button is actually
> > broken. It tries to *download* the page as a resource and fails with "Error
> > downloading: Misdirected Request" which sounds like an artifact of using a
> > redirect to switch from http:// -> https://. I'm not sure if I tried this
> > before, or perhaps I just saw the button and assumed it worked.
> 
> Here the button was not doing anything. Then I restarted ephy and now the
> page is actually loaded as if there was no HSTS in place at all. Not sure
> why this inconsistent behavior, but in this bug I would focus on the API for
> the above.

Disk cache?

[Carlos Garcia Campos]

(In reply to Michael Catanzaro from comment #9)
> (In reply to Claudio Saavedra from comment #7) 
> > I wonder if you could use WebKitWebsiteDataManager to fetch the HSTS data
> > for the site in question. If there is HSTS data you probably want to
> > disallow continuing. This implies checking for every site that fails to load
> > with a SSL error, I guess, so it might not be ideal.
> 
> Hm, maybe a good idea... that might even be the ideal solution.
> 
> Let's do the check at the WebKit level, though. Can't expect dozens of
> different applications to do such a check.
> 
> (In reply to Carlos Garcia Campos from comment #1)
> > hsts-enforced signal is not emitted for that url. Both ff and chromium allow
> > to accept the risk and continue.
> 
> Huh. So Claudio tested Chrome and found you can't continue. I just tested
> Firefox, and was also unable to continue. Could you have typed the URL
> wrong? It is: http://suddomain.preloaded-hsts.badssl.com/

I've copy pasted in both ff and chromium and in both cases I can click in advance button and then proceed. The end result is a 404, though.

> I checked Epiphany again just now and found the continue button is actually
> broken. It tries to *download* the page as a resource and fails with "Error
> downloading: Misdirected Request" which sounds like an artifact of using a
> redirect to switch from http:// -> https://. I'm not sure if I tried this
> before, or perhaps I just saw the button and assumed it worked.

Yes, that happens here too, but not in MiniBrowser.

[Michael Catanzaro]

(In reply to Claudio Saavedra from comment #10)
> Sure, makes sense. What would the API look like though?

Strawman proposal: WebKitWebView::load-failed-with-hsts-error. It would function the same as WebKitWebView::load-failed-with-tls-errors, except we document that calling webkit_web_context_allow_tls_certificate_for_host() will not allow the load to succeed, in contrast to WebKitWebView::load-failed-with-tls-errors. If FALSE is returned (default), then WebKitWebView::load-failed would get called with a network error. (WebKitWebView::load-failed-with-tls-errors would not be called, because that would trick older applications into thinking they can ignore the error.)

[Claudio Saavedra]

How would that implementation work?

I'm assuming that what we're doing is that, regardless of the TLS failure, we check whether there's a HSTS policy, in which case we would emit ::load-failed-with-hsts-error. Then if webkit_web_context_allow_tls_certificate_for_host() gets called, we have to check again if there's a HSTS policy and fail/warn/etc about it.

If that's the case this could work.. the key is that we need to check for every TLS failure regardless of what caused it.

[Michael Catanzaro]

(In reply to Claudio Saavedra from comment #14)
> How would that implementation work?
> 
> I'm assuming that what we're doing is that, regardless of the TLS failure,
> we check whether there's a HSTS policy, in which case we would emit
> ::load-failed-with-hsts-error. Then if
> webkit_web_context_allow_tls_certificate_for_host() gets called, we have to
> check again if there's a HSTS policy and fail/warn/etc about it.

Maybe my proposal is too aggressive in including a behavior change for webkit_web_context_allow_tls_certificate_for_host(). We can leave this unchanged and just document that the browser probably shouldn't call it inside ::load-failed-with-hsts-error because that would violate the spec. I don't think we need to go out of our way to entirely prevent browsers from doing what they want. It should suffice to make it easy to do the right thing by default.

E.g. we don't want to entirely block use of self-signed certificates with HSTS; those are still fine as long as the use is known in advance.

> If that's the case this could work.. the key is that we need to check for
> every TLS failure regardless of what caused it.

Well every TLS failure on a site with HSTS policy would result in the new signal, yes.