155754 – REGRESSION (r191180): Safari does not send Referer Header to iframe src in certain situations

Adam Podolnick

Reported 2016-03-22 10:23:53 PDT

Overview: Safari seems to no longer send a Referer header when loading a page through an iframe in some situations. I'm still trying to isolate the exact cause, but it seems to happen reliably when an external reference, such as a javascript file or css file, is loaded in the document head. Also, the Accept header seems to switch to */* instead of text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Steps to Reproduce: 1) I set up a test case here: https://sproutvideo-examples.s3.amazonaws.com/safari_headers_b.html. View this page in Safari on a device running iOS 9.3 2) Refresh the page if the HTTP_REFERER header is still displayed. It should disappear after a reload. Actual Results: HTTP_REFERER header is missing HTTP_ACCEPT header is */* Expected Results: HTTP_REFERER header should be https://sproutvideo-examples.s3.amazonaws.com/safari_headers_b.html HTTP_ACCEPT header should be text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Platform: iOS 9.3 Other Platforms: Works as expected on iOS 9.2 and below Works as expected on Safari, Chrome, Opera, and Firefox on Mac OSX 10.11.3 Works as expected on Safari, Chrome, Firefox, Opera, Edge, and IE on Windows 10

David Kilzer (:ddkilzer)

Comment 1 2016-03-22 12:04:24 PDT

Does this happen to be a cross-origin request?

Adam Podolnick

Comment 2 2016-03-22 12:05:16 PDT

(In reply to comment #1) > Does this happen to be a cross-origin request? Yes, it is a cross-origin request.

Radar WebKit Bug Importer

Comment 3 2016-03-22 12:05:46 PDT

<rdar://problem/25296445>

Adrian Holovaty

Comment 4 2016-03-24 07:01:16 PDT

We're seeing this same bug with soundslice.com, which offers an embedded iframe sheet-music viewer that optionally does referrer checking. Referrer checking is broken in iOS 9.3, which means our iframes are broken. This bug also affects Vimeo Pro video embeds, which apparently do the same whitelist referrer checking.

Adrian Holovaty

Comment 5 2016-03-24 07:09:52 PDT

Here are people complaining about it on Vimeo's forums: https://vimeo.com/forums/help/topic:281389

Andy Estes

Comment 6 2016-03-25 03:51:59 PDT

I believe this was caused by http://trac.webkit.org/r191180

Brady Eidson

Comment 7 2016-03-25 14:11:01 PDT

(In reply to comment #6) > I believe this was caused by http://trac.webkit.org/r191180 I was thinking the same thing, but the reporter says they see this in iOS 9.3 Did the preload scanner ship in 9.3?

Chris Dumez

Comment 8 2016-03-25 14:12:52 PDT

(In reply to comment #7) > (In reply to comment #6) > > I believe this was caused by http://trac.webkit.org/r191180 > > I was thinking the same thing, but the reporter says they see this in iOS 9.3 > > Did the preload scanner ship in 9.3? The preload scanner shipped in 9.3 but iframe preloading did not ship in 9.3: this is a fairly recent addition.

Brady Eidson

Comment 9 2016-03-25 14:15:33 PDT

(In reply to comment #8) > (In reply to comment #7) > > (In reply to comment #6) > > > I believe this was caused by http://trac.webkit.org/r191180 > > > > I was thinking the same thing, but the reporter says they see this in iOS 9.3 > > > > Did the preload scanner ship in 9.3? > > The preload scanner shipped in 9.3 but iframe preloading did not ship in > 9.3: this is a fairly recent addition. I was thinking of a different thing that we've seen recent issues with (speculative validation) Okay, good.

Chris Dumez

Comment 10 2016-03-25 14:21:30 PDT

(In reply to comment #6) > I believe this was caused by http://trac.webkit.org/r191180 I have just double-checked: http://trac.webkit.org/r191180 / <rdar://problem/23094475> did not ship yet.

Adam Podolnick

Comment 11 2016-03-25 14:22:22 PDT

This might be helpful. If there are no external resources loaded in the <head> or, more generally, before the iframe, the correct headers are sent. Example: https://sproutvideo-examples.s3.amazonaws.com/safari_headers_a.html It also sends the correct headers if the external resource is included AFTER the iframe: https://sproutvideo-examples.s3.amazonaws.com/safari_headers_c.html

Chris Dumez

Comment 12 2016-03-25 15:42:34 PDT

(In reply to comment #10) > (In reply to comment #6) > > I believe this was caused by http://trac.webkit.org/r191180 > > I have just double-checked: http://trac.webkit.org/r191180 / > <rdar://problem/23094475> did not ship yet. Never mind, Andy proved me wrong. r191180 did indeed ship in 9.3 and is likely to cause of this regression.

Ohkawa Yuichi

Comment 13 2016-03-30 21:39:23 PDT

This bug also happen on OS X 10.11.4(Safari 9.1/11601.5.17.1 and Technology Preview 9.1.1). In addition, this also happen with same-origin request. Sample page: https://www.ei.tohoku.ac.jp/demo/sample.html Steps: 1) Access above page. 2) Reload the page. A sample page has style tag and script tag. If there is no style tag(*1) or no script tag(*2) in page, safari sends referer header correctly. Additional samples: *1 https://www.ei.tohoku.ac.jp/demo/sample3.html *2 https://www.ei.tohoku.ac.jp/demo/sample2.html In these sample, I used php page for iframe src. If I used static page for iframe src, the behavior changed. When pages are cached in safari, safari sends referer header. However, if I clear page caches, safari became not to send.

Chris Dumez

Comment 14 2016-03-31 12:53:19 PDT

Created attachment 275313 [details] Roll out

Chris Dumez

Comment 15 2016-03-31 13:27:14 PDT

Created attachment 275315 [details] Patch

WebKit Commit Bot

Comment 16 2016-03-31 14:11:38 PDT

The commit-queue encountered the following flaky tests while processing attachment 275313 [details]: transitions/default-timing-function.html bug 138901 (author: simon.fraser@apple.com) The commit-queue is continuing to process your patch.

Chris Dumez

Comment 17 2016-03-31 14:12:51 PDT

Comment on attachment 275315 [details] Patch Clearing flags on attachment: 275315 Committed r198917: <http://trac.webkit.org/changeset/198917>

Chris Dumez

Comment 18 2016-03-31 14:12:57 PDT

All reviewed patches have been landed. Closing bug.

Adam Podolnick

Comment 19 2016-03-31 14:17:09 PDT

Thanks for looking into this! I see that this has been marked as "RESOLVED FIXED". Has this been tested against the test cases that Ohkawa Yuichi and I provided to make sure that the iframe preloading was the source of the problem?

Chris Dumez

Comment 20 2016-03-31 15:49:32 PDT

(In reply to comment #19) > Thanks for looking into this! I see that this has been marked as "RESOLVED > FIXED". Has this been tested against the test cases that Ohkawa Yuichi and I > provided to make sure that the iframe preloading was the source of the > problem? Yes, I have confirmed that both test cases now work on iOS with latest WebKit.

Adam Podolnick

Comment 21 2016-03-31 15:51:22 PDT

Excellent! Thanks for clarifying.

cp30

Comment 22 2016-04-02 09:52:38 PDT

Hi, We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. First page load is ok but subsequent refreshes drop the referrer once again. Doesn't seem that this one is fixed yet. Thanks

Chris Dumez

Comment 23 2016-04-02 10:18:20 PDT

(In reply to comment #22) > Hi, > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > First page load is ok but subsequent refreshes drop the referrer once again. > Doesn't seem that this one is fixed yet. > > Thanks It is fixed, just not in iOS 9.3.1 yet.

cp30

Comment 24 2016-04-02 10:20:14 PDT

Ah - please accept my apologies - not familiar with the way these things work with Apple updates etc. Thanks for confirming, I will let our users know. Kind regards

Ruudi

Comment 25 2016-04-06 04:46:47 PDT

(In reply to comment #23) > (In reply to comment #22) > > Hi, > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > First page load is ok but subsequent refreshes drop the referrer once again. > > Doesn't seem that this one is fixed yet. > > > > Thanks > > It is fixed, just not in iOS 9.3.1 yet. This problem is still affecting our website after iOS 9.3.1 is installed on device. Please can you advise when you think this bug fix will be sent out in an update by Apple. If the fix is in 9.3.1 then the problem still exists and is not resolved. Thanks

Chris Dumez

Comment 26 2016-04-06 08:49:23 PDT

(In reply to comment #25) > (In reply to comment #23) > > (In reply to comment #22) > > > Hi, > > > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > > First page load is ok but subsequent refreshes drop the referrer once again. > > > Doesn't seem that this one is fixed yet. > > > > > > Thanks > > > > It is fixed, just not in iOS 9.3.1 yet. > > This problem is still affecting our website after iOS 9.3.1 is installed on > device. Please can you advise when you think this bug fix will be sent out > in an update by Apple. If the fix is in 9.3.1 then the problem still exists > and is not resolved. Thanks The bug *is* fixed in WebKit trunk. Yes, the bug still exists in iOS 9.3.1. Apple does not comment on the timing or content of future releases.

Ruudi

Comment 27 2016-04-06 09:28:43 PDT

(In reply to comment #26) > (In reply to comment #25) > > (In reply to comment #23) > > > (In reply to comment #22) > > > > Hi, > > > > > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > > > First page load is ok but subsequent refreshes drop the referrer once again. > > > > Doesn't seem that this one is fixed yet. > > > > > > > > Thanks > > > > > > It is fixed, just not in iOS 9.3.1 yet. > > > > This problem is still affecting our website after iOS 9.3.1 is installed on > > device. Please can you advise when you think this bug fix will be sent out > > in an update by Apple. If the fix is in 9.3.1 then the problem still exists > > and is not resolved. Thanks > > The bug *is* fixed in WebKit trunk. Yes, the bug still exists in iOS 9.3.1. > Apple does not comment on the timing or content of future releases. Thank you - sorry for asking basic questions - I'm not familiar with this. You can understand why I ask, as our site is subscription based and don't want to remove the domain level privacy on our videos. We are trying to understand the time implications so we can manage our customer expectations. If you could indulge me whilst I ask a couple more basic questions: How does the process work once you have resolved the bug fix? Do you submit your bug fix to Apple? DO they normally accept it and use it or are we likely to have this problem in the long term, until they decide the problem is big enough to issue a fix? Thank you in advance for your time - it's most appreciated

Chris Dumez

Comment 28 2016-04-06 11:19:17 PDT

(In reply to comment #27) > (In reply to comment #26) > > (In reply to comment #25) > > > (In reply to comment #23) > > > > (In reply to comment #22) > > > > > Hi, > > > > > > > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > > > > First page load is ok but subsequent refreshes drop the referrer once again. > > > > > Doesn't seem that this one is fixed yet. > > > > > > > > > > Thanks > > > > > > > > It is fixed, just not in iOS 9.3.1 yet. > > > > > > This problem is still affecting our website after iOS 9.3.1 is installed on > > > device. Please can you advise when you think this bug fix will be sent out > > > in an update by Apple. If the fix is in 9.3.1 then the problem still exists > > > and is not resolved. Thanks > > > > The bug *is* fixed in WebKit trunk. Yes, the bug still exists in iOS 9.3.1. > > Apple does not comment on the timing or content of future releases. > > > Thank you - sorry for asking basic questions - I'm not familiar with this. > You can understand why I ask, as our site is subscription based and don't > want to remove the domain level privacy on our videos. We are trying to > understand the time implications so we can manage our customer expectations. > > If you could indulge me whilst I ask a couple more basic questions: > > How does the process work once you have resolved the bug fix? > Do you submit your bug fix to Apple? > DO they normally accept it and use it or are we likely to have this problem > in the long term, until they decide the problem is big enough to issue a fix? > > Thank you in advance for your time - it's most appreciated (In reply to comment #27) > (In reply to comment #26) > > (In reply to comment #25) > > > (In reply to comment #23) > > > > (In reply to comment #22) > > > > > Hi, > > > > > > > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > > > > First page load is ok but subsequent refreshes drop the referrer once again. > > > > > Doesn't seem that this one is fixed yet. > > > > > > > > > > Thanks > > > > > > > > It is fixed, just not in iOS 9.3.1 yet. > > > > > > This problem is still affecting our website after iOS 9.3.1 is installed on > > > device. Please can you advise when you think this bug fix will be sent out > > > in an update by Apple. If the fix is in 9.3.1 then the problem still exists > > > and is not resolved. Thanks > > > > The bug *is* fixed in WebKit trunk. Yes, the bug still exists in iOS 9.3.1. > > Apple does not comment on the timing or content of future releases. > > > Thank you - sorry for asking basic questions - I'm not familiar with this. > You can understand why I ask, as our site is subscription based and don't > want to remove the domain level privacy on our videos. We are trying to > understand the time implications so we can manage our customer expectations. > > If you could indulge me whilst I ask a couple more basic questions: > > How does the process work once you have resolved the bug fix? > Do you submit your bug fix to Apple? > DO they normally accept it and use it or are we likely to have this problem > in the long term, until they decide the problem is big enough to issue a fix? > > Thank you in advance for your time - it's most appreciated Have you tried iOS 9.3.2 beta that was seeded to developers last week?

Chris Dumez

Comment 29 2016-04-06 11:34:03 PDT

(In reply to comment #28) > (In reply to comment #27) > > (In reply to comment #26) > > > (In reply to comment #25) > > > > (In reply to comment #23) > > > > > (In reply to comment #22) > > > > > > Hi, > > > > > > > > > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > > > > > First page load is ok but subsequent refreshes drop the referrer once again. > > > > > > Doesn't seem that this one is fixed yet. > > > > > > > > > > > > Thanks > > > > > > > > > > It is fixed, just not in iOS 9.3.1 yet. > > > > > > > > This problem is still affecting our website after iOS 9.3.1 is installed on > > > > device. Please can you advise when you think this bug fix will be sent out > > > > in an update by Apple. If the fix is in 9.3.1 then the problem still exists > > > > and is not resolved. Thanks > > > > > > The bug *is* fixed in WebKit trunk. Yes, the bug still exists in iOS 9.3.1. > > > Apple does not comment on the timing or content of future releases. > > > > > > Thank you - sorry for asking basic questions - I'm not familiar with this. > > You can understand why I ask, as our site is subscription based and don't > > want to remove the domain level privacy on our videos. We are trying to > > understand the time implications so we can manage our customer expectations. > > > > If you could indulge me whilst I ask a couple more basic questions: > > > > How does the process work once you have resolved the bug fix? > > Do you submit your bug fix to Apple? > > DO they normally accept it and use it or are we likely to have this problem > > in the long term, until they decide the problem is big enough to issue a fix? > > > > Thank you in advance for your time - it's most appreciated > > (In reply to comment #27) > > (In reply to comment #26) > > > (In reply to comment #25) > > > > (In reply to comment #23) > > > > > (In reply to comment #22) > > > > > > Hi, > > > > > > > > > > > > We're still seeing the same issue with iOS 9.3.1 if the page is refreshed. > > > > > > First page load is ok but subsequent refreshes drop the referrer once again. > > > > > > Doesn't seem that this one is fixed yet. > > > > > > > > > > > > Thanks > > > > > > > > > > It is fixed, just not in iOS 9.3.1 yet. > > > > > > > > This problem is still affecting our website after iOS 9.3.1 is installed on > > > > device. Please can you advise when you think this bug fix will be sent out > > > > in an update by Apple. If the fix is in 9.3.1 then the problem still exists > > > > and is not resolved. Thanks > > > > > > The bug *is* fixed in WebKit trunk. Yes, the bug still exists in iOS 9.3.1. > > > Apple does not comment on the timing or content of future releases. > > > > > > Thank you - sorry for asking basic questions - I'm not familiar with this. > > You can understand why I ask, as our site is subscription based and don't > > want to remove the domain level privacy on our videos. We are trying to > > understand the time implications so we can manage our customer expectations. > > > > If you could indulge me whilst I ask a couple more basic questions: > > > > How does the process work once you have resolved the bug fix? > > Do you submit your bug fix to Apple? > > DO they normally accept it and use it or are we likely to have this problem > > in the long term, until they decide the problem is big enough to issue a fix? > > > > Thank you in advance for your time - it's most appreciated > > Have you tried iOS 9.3.2 beta that was seeded to developers last week? Ok, I have verified that I cannot reproduce the bug anymore on the public iOS 9.3.2 beta that was seeded to developers last week. I confirmed using both test cases (the one from Adam and the one from Ohkawa) on this bug report. Hopefully this helps. Feel free to confirm it yourself as well.

Ruudi

Comment 30 2016-04-06 12:24:20 PDT

Thank you !

Gabriel Barros

Comment 31 2016-05-23 17:36:04 PDT

I know there is no comment on timing or content of future releases, but where can I subscribe/be informed when the release for OSX happens? There will be no changelog mentioning this bug number, I assume since i did not see references to bug numbers in any other changelog. Will this page be updated when it happens? Sorry for not being familiar with the process. Feel free to just point me to a FAQ if there is one I failed to find.

Brady Eidson

Comment 32 2016-05-23 17:48:14 PDT

(In reply to comment #31) > I know there is no comment on timing or content of future releases, but > where can I subscribe/be informed when the release for OSX happens? No such notification exists for OS X Safari releases. If you're interested in Safari Technology Preview releases, the release notes for those have - so far - included ChangeLogs referencing bugzillas. > Will this page be updated when it happens? No.

Gabriel Barros

Comment 33 2016-05-23 18:18:42 PDT

> If you're interested in Safari Technology Preview releases, the release notes > for those have - so far - included ChangeLogs referencing bugzillas. Thank you. Can I assume the reference on the release-notes[1] will be the exact (or similar) title here? [1] https://developer.apple.com/safari/technology-preview/release-notes/

Chris Dumez

Comment 34 2016-05-23 18:27:04 PDT

(In reply to comment #31) > I know there is no comment on timing or content of future releases, but > where can I subscribe/be informed when the release for OSX happens? > > There will be no changelog mentioning this bug number, I assume since i did > not see references to bug numbers in any other changelog. > > Will this page be updated when it happens? > > Sorry for not being familiar with the process. Feel free to just point me to > a FAQ if there is one I failed to find. While there is no comment on the content of future releases. I am pretty sure the fix already shipped in 10.11.5 on May 16.

Attachments
Roll out (5.57 KB, patch) 2016-03-31 12:53 PDT, Chris Dumez	no flags	Details Formatted Diff Diff
Patch (7.75 KB, patch) 2016-03-31 13:27 PDT, Chris Dumez	no flags	Details Formatted Diff Diff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.