Bug 114738 – 304s broken for main resources

• Status: UNCONFIRMED
• Component: Page Loading

[Alexander Romanovich]

A great deal of work was done recently by Nate Chapin on the caching of main resources in memory cache. Ideally this would lead to Safari finally supporting 304 responses on main resource loads, but that's still not the case. I believe this is a bug in the current implementation.

Here is an example PHP script that sends a few headers to demonstrate the issue: http://www.sirensclef.com/webkit/304/example.php
304s are supported in Chrome and FF on that page.

Interestingly, I found this page in bugzilla which partially works: http://renevier.net/misc/webkit_109225.php
I get a 200 the first time, and each subsequent refresh. However, if I manually press enter from WebKit nightly's URL field, it gives a 304 and then continues to give a 304 on each refresh.

Lastly, here is a plain HTML page that gets 200 in Safari, but 304 in Chrome/FF: http://www.sirensclef.com/webkit/304/index.html

Can this please be fixed now? After the recent work that was done, I would think this is a great opportunity to finally deliver some basic functionality that Safari users & developers have been waiting years for.

[Brady Eidson]

This report is difficult for me to follow.  It describes a lot of history and a lot of different behaviors.

But it doesn't give a succinct description of the steps to reproduce the bug, what the results of those steps are, and what the expected behavior is.

Even the title - "304s broken for main resources" - doesn't really describe anything...  I can view main resources that were revalidated with a 304 response just fine!  In what way are they broken?

[Brady Eidson]

Could you try with a format like:

* STEPS TO REPRODUCE
1. Setup or prep work
2. Explicit steps to reproduce
3. And only the steps that are necessary.

* RESULTS
The observed behavior

* EXPECTED RESULTS
The expected behavior, highlight how it differs from the observed behavior

[Alexander Romanovich]

* STEPS TO REPRODUCE
1. Launch WebKit nightly.
2. Load url: http://www.sirensclef.com/webkit/304/index.html
3. Open Web Inspector.
4. Hit cmd-r repeatedly. Note that it gives a 200 response every time.
5. Launch Chrome.
6. Load and inspect the same url.
7. Refreshing gives 304 every time.

* RESULTS
Safari gives a 200 response every time.

* EXPECTED RESULTS
Chrome, Firefox both get 304 responses each time the page is refreshed.

The same holds true with the other sample urls I have provided.

[Brady Eidson]

(In reply to comment #3)
> * STEPS TO REPRODUCE
> 1. Launch WebKit nightly.
> 2. Load url: http://www.sirensclef.com/webkit/304/index.html
> 3. Open Web Inspector.
> 4. Hit cmd-r repeatedly. Note that it gives a 200 response every time.
> 5. Launch Chrome.
> 6. Load and inspect the same url.
> 7. Refreshing gives 304 every time.
> 
> * RESULTS
> Safari gives a 200 response every time.
> 
> * EXPECTED RESULTS
> Chrome, Firefox both get 304 responses each time the page is refreshed.
> 
> The same holds true with the other sample urls I have provided.

So what you're saying is "I wish cmd-R in Safari did a revalidate instead of an origin reload", correct?

If so, that's probably an enhancement request for Safari and not a bug report for WebKit.

[Alexander Romanovich]

Oh yes, I got involved in that request 3 years ago here: https://bugs.webkit.org/show_bug.cgi?id=17998

Gosh, this "enhancement request" was first reported 5 years ago. From the discussion there, it seems not everyone agrees it isn't a bug. I guess the question remains: can that issue be revisited now with the new memory caching going on?

[Brady Eidson]

(In reply to comment #5)
> Gosh, this "enhancement request" was first reported 5 years ago. From the discussion there, it seems not everyone agrees it isn't a bug.

In general, snarky attitude doesn't help make a bug report in the WebKit project more important.

My understanding was that you were requesting a change in behavior in Safari - Which is why I *directly asked* if that's what you were requesting.

A change in behavior in Safari would have to be filed as a bug report with Apple, and in Apple bug parlance "enhancement request" is an actual categorization of bug report that is asking for a change in behavior when the current behavior is not obviously wrong.

That said...

> Oh yes, I got involved in that request 3 years ago here: https://bugs.webkit.org/show_bug.cgi?id=17998
> 

Thank you for providing that link.  In the future, if you are filing a follow up bug to a previous bug, it would be extremely helpful to include that context up front and would help avoid a lot of confusion.

So my understanding now is that if you hit cmd-R in Safari (with a WebKit nightly) on a page that includes subresources... that the subresources are requested with a conditional GET but the main resource is still requested with a non-conditional GET.

Is that the bug you are reporting?

[Alexander Romanovich]

Apologies Brady, that wasn't intended to be snark, just surprise. I think in general it's not well understood among the user community about why this issue isn't being given higher priority, as the original bug discussion illustrates. For myself, I deal with a large customer base with high numbers of Safari users, so there's a lot of wasted bandwidth going on here.

To answer your question: yes, indeed this is the change I am requesting. Though for what it's worth I would put forth a general objection to this being treated as a feature request and hope it would be considered as a true bug. That's your call of course, but my $0.02.

I had forgotten about the other bug at the time I posted this, so feel free to close it now as a duplicate of that one.

[Brady Eidson]

(In reply to comment #7)
> I think in general it's not well understood among the user community about why this issue isn't being given higher priority, as the original bug discussion illustrates. 

Normally for a bug to gain visibility solely because of high demand from "the community" there needs to be much more discussion, many more dupes (both here and in Radar at Apple), people reaching out directly to an evangelist, or *some other indication* that developers want this resolved.

All we have on this is a single bug report on bugs.webkit.org, only 12 people CC'ed, with zero activity since 2010 (your Jan 30th comment aside).

> To answer your question: yes, indeed this is the change I am requesting. Though for what it's worth I would put forth a general objection to this being treated as a feature request and hope it would be considered as a true bug. That's your call of course, but my $0.02.

It's easy to see it as a bug when it is spelled out that subresources are revalidated while the main resource is not.

Again, back to my original comment on this, that wasn't clear at the time with your test case and without the context of the original bugzilla.

> I had forgotten about the other bug at the time I posted this, so feel free to close it now as a duplicate of that one.

Yup, and I've added this information there.

[Brady Eidson]

*** This bug has been marked as a duplicate of bug 17998 ***

[Alexey Proskuryakov]

> 4. Hit cmd-r repeatedly. Note that it gives a 200 response every time.

So you are observing the behavior through Web Inspector? That won't necessarily give you sufficient insight into what's going on with caching. CFNetwork does not return 304 results to the client, it returns cached 200 responses without any indication that they were served from cache.

We can only know that a response is 304 when it's served by WebCore cache.

I think that we have some bugs about this Web Inspector issue, but I can't find them now.

I'm not confident if this bug got resolved correctly. Can you try checking current behavior with tcpdump or equivalent?

[Brady Eidson]

Good catch - I missed that the web inspector was explicitly mentioned here.

We shouldn't assume this is a dupe of the original revalidation bug unless we have actual TCP dumps confirming it, or at the very least server logs showing that the GET was unconditional.

[Alexander Romanovich]

Hi guys. If it helps, this is what the server log shows for a refresh of the example PHP page and the example HTML page. A 200 is logged for Safari and 304 for Chrome.

PHP page:

Safari: [22/Apr/2013:12:25:38 -0400] "GET /webkit/304/example.php HTTP/1.1" 200 31 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.38+ (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

Chrome: [22/Apr/2013:12:26:03 -0400] "GET /webkit/304/example.php HTTP/1.1" 304 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.56 Safari/537.36"

HTML page:

Safari: [22/Apr/2013:12:27:59 -0400] "GET /webkit/304/ HTTP/1.1" 200 57 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.38+ (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

Chrome: [22/Apr/2013:12:28:04 -0400] "GET /webkit/304/ HTTP/1.1" 304 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.56 Safari/537.36"

[Brady Eidson]

Well, by server logs, I'd meant dumping the full set of request headers.  That's also why a TCP dump would be effective - a sniff of the traffic without a logging mechanism masking what's really going on.

[Balazs Kelemen]

Alexander, could you make up a layout test for this? I think you can find sufficient patterns to test this among the http tests (http://trac.webkit.org/browser/trunk/LayoutTests/http/tests), something like testRunner.dumpResourceLoadCallbacks could do the trick.

[Alexander Romanovich]

I lack the time/knowledge for layout tests on this bug. Reached out to another developer following the matter but doesn't appear they've contributed anything. So open invitation to anyone who wants to give it a shot.

[Alexander Romanovich]

Note that with Chrome 56, the behavior between Chrome and Safari has diverged even further.

It appears that now Chrome:

- Revalidates only the main resource on refresh. (https://blog.chromium.org/2017/01/reload-reloaded-faster-and-leaner-page_26.html)
- Revalidates all other resources with a shift-refresh.
- Supports 304s on the main resource (as before).

And Safari:

- Revalides all resources on refresh (i.e. always shift-refreshes).
- Still does not support 304s on the main resource.

I wonder if support for 304s on main resources could be given attention again in the context of the broader context of Chrome's recent decision, and the fact that Safari does not support shift-refresh?

[Jon Adams]

for what it's worth if main resources are initially loaded via responses providing `Cache-Control: must-revalidate` headers it appears that 304s are supported/utilized as expected.

[Alexander Romanovich]

I cannot confirm Jon Adams' comment. In my tests, must-revalidate on the original request does not result in 304s upon subsequent normal reloads in Safari.

Additionally, since Safari DOES now support command-option-refresh (impacting refresh behavior of non-main resources), I'm not sure why Safari cannot at this point move to match all other browsers and perform a revalidate on reload for the main resource as well, and reload from origin only when that's explicitly requested.

[Alexander Romanovich]

Another side effect of this issue is that developers are checking for the cache control header value of "max-age=0" to detect an end user refresh in order to perform certain tasks. They can do that for Chrome/FF/IE but for Safari user agents it requires a process of tracking the last visited page in a cookie or session and then assuming a refresh if the same page is hit again, which is neither the same thing nor efficient.