Bug 27802 – Encoding override is not applied to all the contents in the page.

• Status: NEW
• Component: Page Loading

[johnnyding]

(Original bug was filed by Jungshik)
Ideally, When 'character encoding' is overriden manually by a user (in View > Text Encoding menu), that encoding should be applied to all contents and frames (main and subframes loaded in a few different ways - JS, iframe, frame, AJAX). But now WebKit's behavior is not.
Jungshik described a testcase in chromium bug: http://code.google.com/p/chromium/issues/detail?id=7243

After analyzing the case, some contents don't change is because those contents are generated by external JS file, and the encoding in the C-T head of the external JS files is "EUC-KR". Now if there is encoding defined in C-T head of external JS resource, WebKit will use it to decode the contents of JS resource. 
(See Loader::Host::didReceiveResponse and CacheScript::setEncoding).

To comply with IE's behavior, it looks like we need to give high priority to the override encoding when decoding the external js resource. But now FF3 preferntially tries to use the encoding declared in C-T header if it's available, like current WebKit does. 

If we decide to use IE's way. I have a patch to make it work.

[johnnyding]

Created attachment 33713 [details]
patch v1 for loader

[Alexey Proskuryakov]

Comment on attachment 33713 [details]
patch v1 for loader

This looks closely related to bug 22952, probably even a duplicate. It's definitely a bug, as WebCore caching shouldn't affect how pages look.

But I don't think that the fix is sufficient - it changes the globally cached resource text when encoding override is in effect, but other documents may be using the same resource with other encodings, so they will be broken.

E.g., if you open a page, set an encoding override, and then open the same page in a new window, subresources will be taken from cache with a wrong (overridden) encoding.

[johnnyding]

Thanks,  Alexey. You are right, I shouldn't change the encoding of the global cached resource.  A way comes to my mind to fix this issue is to 
1) Firstly check the whether override encoding of revalidate resource is equal with the existing encoding of the base cached source(m_resourceToRevalidate). If yes, we done.
2) If no, we keep the revalidate resource and copy the data from base cached source. Then change the encoding of revalidate resource to override encoding.

Is the way OK?

[johnnyding]

Created attachment 33775 [details]
patch v2 for loader

Alex, the patch v2 used the way I mentioned in comments #3. Please take a look.
Thanks!

[johnnyding]

Please ignore my patch, I found a problem, will send the patch v2 later.
Sorry for inconvenience.

[Alexey Proskuryakov]

Comment on attachment 33775 [details]
patch v2 for loader

Clearing review flag per the above comment.

[Alexey Proskuryakov]

By the way, if your patch will also fix bug 22952, I'd suggest posting it there, as it's much older.