URI fragment of location header not appended in case of further redirect

• Component: Page Loading
• Status: RESOLVED WONTFIX

[Thomas Gerlach]

Created attachment 280593 [details]
simple php test scenario

As a variant of the BUG 24175 the following scenario still fails in Safari 9 (iPhone 9, Mac 10)

1. request a URI
2. response is 302 with a URI with fragment as location header
3. the location header's URI is redirected again with 302
4. the last URI has no fragment anymore.

example:

1. request:  http://localhost/fragmenttest
2. response: 302, location header /fragmenttest/old/#somewhere
3. response: 302, location header /fragmenttest/new/
4. FAIL: agent requests just http://localhost/fragmenttest/new/ instead of http://localhost/fragmenttest/new/#somewhere

important:

This behaviour is only reproducable with the two redirects.
If you enter the 2. URI directly in the address line of the browser, the fragment will be kept!
2' request: http://localhost/fragmenttest/old/#somewhere
3. response: 302, location header /fragmenttest/new/
4. SUCCESS: agent requests http://localhost/fragmenttest/new/#somewhere


test:

insert the attached folder "fragmenttest" into a webserver with php and call the above mentioned URIs.
(index.php and index.html need to be the configured as default files for folders)

[Brady Eidson]

rdar://problem/26654828

[Scott]

This a fairly critical bug as it impacts page fragment handling in cases where a single sign-on server is used, especially OpenID Connect. Additionally, latest versions of Firefox, Chrome, and Internet Explorer do not exhibit this bug. This leaves web software vendors either having to not support Safari or invent work arounds.

[Alexey Proskuryakov]

*** Bug 170058 has been marked as a duplicate of this bug. ***

[Michael Catanzaro]

Note: Carlos says r214807 fixes this for the soup backend.

[Michael Catanzaro]

Looks like he added a layout test there that's failing on Mac/iOS and marked against this bug. Hopefully that helps a bit with writing a fix.

[Alex Christensen]

Created attachment 306291 [details]
Patch

[Brady Eidson]

Comment on attachment 306291 [details]
Patch

I think we need to do a little more exploration first.

[Brady Eidson]

(In reply to Brady Eidson from comment #7)
> Comment on attachment 306291 [details]
> Patch
> 
> I think we need to do a little more exploration first.

To elaborate, I'm not sure *always* doing this is necessarily right.
e.g.

What if the redirect is cross domain?
What if the redirect is cross protocol (http -> https, or vice versa)?
What if the redirect is a different status code from 302?

I don't know that there's a single, pretty spec that covers this, but we should have a matrix of tests covering all of those cases, verify the other browsers agree, and then then go for it.

[Michael Catanzaro]

Hm, good point Brady.

[Scott]

RFC 7231, section 7.1.2

https://tools.ietf.org/html/rfc7231#section-7.1.2

   If the Location value provided in a 3xx (Redirection) response does
   not have a fragment component, a user agent MUST process the
   redirection as if the value inherits the fragment component of the
   URI reference used to generate the request target (i.e., the
   redirection inherits the original reference's fragment, if any).

[Alex Christensen]

Given that it is a part of the HTTP spec, we should probably fix this in CFNetwork for non-WebKit clients.  Now the decision of whether we want to cover up this bug with this patch in WebKit...

[Brady Eidson]

(In reply to Scott from comment #10)
> RFC 7231, section 7.1.2
> 
> https://tools.ietf.org/html/rfc7231#section-7.1.2
> 
>    If the Location value provided in a 3xx (Redirection) response does
>    not have a fragment component, a user agent MUST process the
>    redirection as if the value inherits the fragment component of the
>    URI reference used to generate the request target (i.e., the
>    redirection inherits the original reference's fragment, if any).

The HTTP spec doesn't seem to distinguish between the concerns I mentioned, but I still wonder if the other browsers do.

[Scott]

(In reply to Brady Eidson from comment #12)
> (In reply to Scott from comment #10)
> > RFC 7231, section 7.1.2
> > 
> > https://tools.ietf.org/html/rfc7231#section-7.1.2
> > 
> >    If the Location value provided in a 3xx (Redirection) response does
> >    not have a fragment component, a user agent MUST process the
> >    redirection as if the value inherits the fragment component of the
> >    URI reference used to generate the request target (i.e., the
> >    redirection inherits the original reference's fragment, if any).
> 
> The HTTP spec doesn't seem to distinguish between the concerns I mentioned,
> but I still wonder if the other browsers do.

Chrome & Firefox:

HTTP -> HTTPS: fragment is preserved
Cross domain: fragment is preserved
Preservation happens for all 3xx redirection

[Brady Eidson]

(In reply to Scott from comment #13)
> (In reply to Brady Eidson from comment #12)
> > (In reply to Scott from comment #10)
> > > RFC 7231, section 7.1.2
> > > 
> > > https://tools.ietf.org/html/rfc7231#section-7.1.2
> > > 
> > >    If the Location value provided in a 3xx (Redirection) response does
> > >    not have a fragment component, a user agent MUST process the
> > >    redirection as if the value inherits the fragment component of the
> > >    URI reference used to generate the request target (i.e., the
> > >    redirection inherits the original reference's fragment, if any).
> > 
> > The HTTP spec doesn't seem to distinguish between the concerns I mentioned,
> > but I still wonder if the other browsers do.
> 
> Chrome & Firefox:
> 
> HTTP -> HTTPS: fragment is preserved
> Cross domain: fragment is preserved
> Preservation happens for all 3xx redirection

That's great: We should make sure we have layout tests for all of those cases.

[Brian Schrader]

Hello maintainers,

Has there been any update on a fix for this bug? It looks like this ticket, and it's predecessor, have been open for a while. I develop an application that was running into this bug and we've had to workaround it for now.

We have an OAuth system that requires a user to be redirected (with a login token fragment), after successful login, to a URL that checks for additional actions the user needs to take before redirecting them again to the web-app itself. Because of this bug, that intermediary check causes the fragment to be stripped and the web-app cannot see that the user is logged in.

For what it's worth, this bug doesn't seem to be affecting certain macOS clients. I was originally able to reproduce the issue on my development machine (running High Sierra), but after a few failed login cycles, the fragments are successfully passed (not sure if this is because the Dev Tools are open or not), and once Safari successfully passed the fragments once, it does so every single subsequent time. Others who are not developers are never able to get through the checks, and iOS users consequently can never log in.

I'd appreciate any help or information about this bug and a fix, and I'm happy to provide more info if needed.

Thanks.

[Thierry allard saint albin]

Hello,

Has there any update on this subject, does someone have a workaround ?

[fetis]

I think we faced this issue as well with Auth0 auth-provider. After successful login Auth0 redirects you with 302 to your configured callback url (URL fragment contains issued token) and we redirect from generic callback to a specific language version.

What I still don't understand, the problem triggers not every time but only sometimes and this sometimes I still can't get. 

Thomas, maybe you know some additional conditions to trigger the bug?

[Tim]

Just to give an example of the impact of this from a downstream POV:

> It's worth considering that (when handling a web-system that uses #fragments), the browsers only agree on what to do if you limit request-handling to a maximum of 1 redirect. Multiple redirects are not portable. That's a very annoying realization. Long-term, it basically means either (a) we regulate to keep the overall system within budget ("maximum 1 redirect"), or (b) we get out of the #fragment business, or (c) Apple fixes Safari.

(from https://github.com/civicrm/civicrm-core/pull/17422)

[David Bokan]

> What if the redirect is cross domain?
> What if the redirect is cross protocol (http -> https, or vice versa)?
> What if the redirect is a different status code from 302?
> 
> I don't know that there's a single, pretty spec that covers this, but we should have a matrix of tests covering all of those cases, verify the other browsers agree, and then then go for it.

FWIW - I've added a web platform test that covers all these cases and a few more: https://wpt.fyi/results/fetch/redirect-navigate/preserve-fragment.html?label=experimental&label=master&aligned

It captures the bug here - where there's multiple redirects, Safari loses the fragment provided in the intermediate/first redirect. Both Chromium and Firefox keep the fragment, as has been pointed out in other comments here.

[Alex Christensen]

All NSURLConnection and NSURLSession requests behave this way, and there isn't a good way to work around this in WebKit.  I filed rdar://problem/65508577 and further investigation will be done there.

David, thanks for making a web platform test for this.  It makes it easy to see what needs to change.

[Brian Schrader]

Glad to hear there's progress being made on this, but I'm sad to see this bug moved to an internal process. We here on the outside can't see or track progress of radars. Will you be posting here when the feature is merged into NSURLConnection/NSURLSession? We've been following this thread for years now and I'm sure we'd all really appreciate knowing it's being worked or completed.

[Alex Christensen]

I believe this is fixed in Big Sur / iOS 14.

[David Bokan]

Great to hear, thanks Alex!

One question: On WPT.fyi the test still shows failures for the last case, where we have a redirect chain that looks like: urlA#target -> urlB#fromRedirect -> urlC. In this case, Firefox and Chrome both use #fromRedirect (which is correct based on my read of https://tools.ietf.org/html/rfc7231#section-7.1.2) but Safari still keeps #target.

WPT says that's running on Safari 116 on macOS 10.15, is that still broken or does it just need to update to a newer version of Safari/Mac?

[Alex Christensen]

It is still broken on macOS 10.15.

[David Bokan]

Oops, I mixed up the iOS and Mac version numbering.

Thanks!