Bug 46573 - REGRESSION: charset="utf-8" (with quotes) doesn't work
Summary: REGRESSION: charset="utf-8" (with quotes) doesn't work
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Page Loading (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P1 Normal
Assignee: Alexey Proskuryakov
URL:
Keywords: InRadar, Regression
Depends on:
Blocks:
 
Reported: 2010-09-25 14:16 PDT by Max Barel
Modified: 2010-12-12 00:06 PST (History)
6 users (show)

See Also:


Attachments
proposed fix (4.83 KB, patch)
2010-12-10 15:38 PST, Alexey Proskuryakov
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Max Barel 2010-09-25 14:16:51 PDT
Beginning with r64889, any html code received from XHR is incorrectly interpreted against latin1, ignoring the "charset="utf-8"'directive in Content-Type.
The last well behaving release is r64816

This bug have been there years ago and corrected.

BTW. I'm pretty sure I already reported this bug (I mean with this very release information) but can't find it anymore.
Comment 1 Alexey Proskuryakov 2010-09-25 21:56:32 PDT
Could you please provide an example page, or a reduced test case? Does this page work in Firefox?

The only suspicious change in this range is one where we made charset name parsing more strict. If a resource used something weird like "utf_8" or "u-t-f-8", then we'd no longer treat it as utf-8. But common forms are all supposed to work.
Comment 2 Max Barel 2010-09-26 04:18:40 PDT
Alexey,
The test page was working in Firefox.
Nevertheless, your comment directed me to the point.
I was serving the header as: Content-type: text/html; charset="utf-8"
rather than : Content-type: text/html; charset=utf-8

Notice that the encoding was a quoted string in the faulty page.

The Content-type specification (http://www.w3.org/Protocols/rfc1341/4_Content-Type.html) allows quoted strings for parameter values, but I'm not sure the specific charset parameter does. Maybe not.

So, it's up to you to decide if the parsing is indeed too strict. Should we allow quoted strings there?
Beside, you may close the bug.
Comment 3 Alexey Proskuryakov 2010-09-26 10:19:34 PDT
Thank you, this is certainly a bug. It's not even limited to XMLHttpRequest, in fact.

Technically, the bug is in closed source networking code used by WebKit, but we'll need to add a workaround.
Comment 4 Alexey Proskuryakov 2010-09-26 10:20:33 PDT
<rdar://problem/8479389>
Comment 5 Alexey Proskuryakov 2010-12-10 15:38:08 PST
Created attachment 76270 [details]
proposed fix

Firefox also seems to support single quotes, but IE doesn't. RFC 2616 also seems to only allow double quotes.
Comment 6 WebKit Commit Bot 2010-12-10 19:31:54 PST
Comment on attachment 76270 [details]
proposed fix

Rejecting attachment 76270 [details] from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=eseidel-cq-sl', 'build', '--no-clean', '--no-update', '--build-style=both']" exit_code: 2
Last 500 characters of output:
/CommitQueue/WebKitBuild/WebKit.build/Debug/WebKit.build/Objects-normal/x86_64/WebPluginContainerCheck.o /Projects/CommitQueue/WebKit/mac/Plugins/WebPluginContainerCheck.mm normal x86_64 objective-c++ com.apple.compilers.gcc.4_2
	CompileC /Projects/CommitQueue/WebKitBuild/WebKit.build/Debug/WebKit.build/Objects-normal/x86_64/WebTextCompletionController.o /Projects/CommitQueue/WebKit/mac/WebView/WebTextCompletionController.mm normal x86_64 objective-c++ com.apple.compilers.gcc.4_2
(27 failures)


Full output: http://queues.webkit.org/results/6900032
Comment 7 Alexey Proskuryakov 2010-12-10 20:56:11 PST
Comment on attachment 76270 [details]
proposed fix

What?
Comment 8 Eric Seidel (no email) 2010-12-10 21:01:12 PST
This machine is having trouble with XCode.  It's unclear why.  I'll see if I can blow some caches or something.
Comment 9 WebKit Review Bot 2010-12-10 22:15:37 PST
Comment on attachment 76270 [details]
proposed fix

Clearing flags on attachment: 76270

Committed r73836: <http://trac.webkit.org/changeset/73836>
Comment 10 WebKit Review Bot 2010-12-10 22:15:42 PST
All reviewed patches have been landed.  Closing bug.
Comment 11 WebKit Review Bot 2010-12-10 22:47:26 PST
http://trac.webkit.org/changeset/73836 might have broken Leopard Intel Release (Build) and Leopard Intel Debug (Build)
Comment 12 Alexey Proskuryakov 2010-12-10 23:22:33 PST
What?

Fixed the build in r73844.
Comment 13 Alexey Proskuryakov 2010-12-12 00:06:58 PST
I was obviously asleep at the wheel, and made a change to ResourceResponseCFNet.cpp instead of a Mac version. Thanks Alice for fixing it properly!