Bug 14665 – WebKit loads the wrong resource files

• Status: UNCONFIRMED
• Component: Page Loading

[Rush Manbert]

This bug exists in both the nightly build and in the currently shipping WebKit that comes with Mac OS X 10.4.10. My Safari version says that it is version 2.0.4 (419.3), where I assume that the WebKit version is 419.3. The following description duplicates the Radar bug report that I filed with Apple. It describes the behavior observed when the test program is run with the WebKit that comes with 10.4.10.

***** Duplicate of Apple Report *****
Summary: 
I am using WebKit to load HTML content from a file. The HTML file contains a resource reference for an image  file. My application allows the user, via configuration data, to substitute the referenced image file with a different image file. I handle this by intercepting the
-(NSURLRequest *)webView:(WebView *)sender resource:(id)identifier 
	willSendRequest:(NSURLRequest *)request redirectResponse:(NSURLResponse *)redirectResponse
		fromDataSource:(WebDataSource *)dataSource
delegate callback. In that method, I use the original image filename as a dictionary key and select a different image filename. I then construct a new NSURLRequest from the file path and return it in place of the original request. This all works. The problem is that EACH time I load the HTML file I need to select the proper image file to load, and it can change each time. However, after the first time I pick a file and load it, the WebKit always loads that same resource file, no matter what I tell it to do differently.

I have experimented with specifying cache settings on the NSURLRequests, and they make no difference. The wrong resource gets loaded.

Steps to Reproduce: 
I have included a Xcode project that demonstrates the problem. It started out as the Special Picture Protocol sample program from Apple. I have modified it to do the following:
1) It initially loads a file called test.html. At the start of the program and every time the Reset button is pressed, it resets a page load counter value to zero and loads test.html.
2) The test.html file loads an image file called test.jpg.
3) In the webView: willSendRequest delegate callback I look for a request for test.jpg. When I see it, I replace the NSURLRequest with a new one. The new request is for either even.jpg or odd.jpg, depending on whether the page load counter is an odd or even value.
4) The replacement NSURLRequest is logged to the console and the page load counter is incremented.
5) The web page contains a link that reloads test.html. Each time it reloads, we alternate the image resource file selection.

All you need to do is build and execute the test application. Once the test.html page is displayed, click on the link to load it again. Note that the image file is always even.jpg, and is never odd.jpg.

Expected Results: 
I expected that the image file content would alternate between the contents of even.jpg and odd.jpg

Actual Results: 
What actually happens is that whatever resource file was selected the first time is always used, no matter what the replacement NSURLRequest says.

You can demonstrate this by editing line 104 in MyController.m If you change it to say:
pageLoadCounter = 1;

and rebuild the app, you will see that the first time through we load the odd.jpg image file, and we never see even.jpg.

You can also see there are commented out lines that add the NSURLRequestReloadIgnoringCacheData cache policy. This can be done in the original (test.html) request, and in the replacement (even.jpg/odd.jpg) requests, but it doesn't change the bad behavior.

Regression: 
My real application has multiple independent WebViews open. In that application, I actually allow my users to localize any resource reference, and I provide a path registration mechanism that lets me find resources in different locations, even if the resource filename is not modified. I first saw this problem because I had two independent WebViews open, and each was running the same web application, but localized different ways. What I saw was that both of the windows were loading the image resources that were first loaded for a particular HTML page. It just mattered which WebView displayed the HTML page first and how its images were localized. Then those were used by the other WebView, even though we selected different image files for display.

If I duplicate my application and run two separate application instances, then this problem does not occur. Each application instance acts independently and displays the proper data.

It certainly seems that something is being cached, and its key seems to be the original resource speciication that I am replacing. That's the only explanation I can come up with for why the same resource file gets loaded over and over.

Notes: 
The attached file "WrongResourceFileLoaded.zip" is an archive of my Xcode project directory.


'WrongResourceLoaded.zip' was successfully uploaded

12-Jul-2007 04:56 PM Rush Manbert:
I tried to work around this problem by writing my own custom file protocol handler. I modified my
-(NSURLRequest *)webView:(WebView *)sender resource:(id)identifier 
	willSendRequest:(NSURLRequest *)request redirectResponse:(NSURLResponse *)redirectResponse
		fromDataSource:(WebDataSource *)dataSource
method so that it ALWAYS creates a new NSURLRequest, even if the original request was for a local file using the file:// protocol. In these cases, I return a new request that specifies myfile://localhost//path/to/file as the URL. What I see is that if the original URL has not been seen, then my file protocol handler gets called for the replacement NSURLRequest. However, once a file has been loaded for the original request, then another request for the same resource file that gets replaced by a DIFFERENT replacement request does NOT call my protocol handler. Something just returns the file content that was previously loaded and skips even calling my handler. That is a major problem and is going to cause me big problems with my customers.

***** End duplicated Apple bug report *****

I have since downloaded the sources, built WebKit, and debugged it. The basic problem is that WebCore::DocLOader::requestResource() calls WebCore::Cache::requestResource() with the original URL. That is used as the hash map key, the resource is not found in the cache, and the cache starts oading it. This eventually calls my delegate method that changes the URL. The code detects that I have changed the URL, but it adds the CachedResource to the cache hash map using the original URL as the key. The next time we try to load that same original URL, the cached resource is found and used. This despite the fact that my delegate method, if called, would change the URL to something different than what it did the first time. The result is that the wrong resource is loaded. This specific problem is happening with images. My guess is that there is a similar problem with other resources.

I believe that a better caching approach would be to check for a cache hit using the original URL as the key, but when this results in a miss, you need to call my willSendRequest delegate and see if I changed the URL. You then need to use the new URL as the cache key value. When a subsequent request comes in for the original URL that results in a miss, you need to call my willSendRequest delegate and see if the URL changes. At that point you need to check the cache again using the replaced URL. This will result in a hit after the first time through.

I have tried to find a surgical way to change the cache behavior, but the resource loading is all tied up with the cache, so it's not very adaptable.

One thing to note: If I test against the WebKit that is in 10.4.10, my willSendRequest delegate gets called every time, even if the wrong resource gets  loaded from the cache. The call stack also looks completely different from what is seen if I run against the WebKit built from current sources. In that case, my willSendRequest delegate only gets called the first time for any image resource. After that, the (incorrect) image is just loaded from the cache and the willoSendRequest delegate is not called.

The Apple Radar bug for this problem is #5315820.

[Rush Manbert]

Created attachment 15570 [details]
Sample project that demonstrates the caching problem.