Bug 28685

Summary: Web Inspector: Make localizedString.js diff and commit friendly (UTF16 -> UTF8)
Product: WebKit Reporter: Patrick Mueller <pmuellr>
Component: Web InspectorAssignee: Joseph Pecoraro <joepeck>
Status: RESOLVED FIXED    
Severity: Normal CC: bburg, commit-queue, joepeck, mattbaker, mitz, mrowe, timothy, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: 528+ (Nightly build)   
Hardware: All   
OS: All   
Attachments:
Description Flags
[PATCH] Proposed Fix none

Description Patrick Mueller 2009-08-24 13:27:20 PDT
The file WebCore/English.lproj/localizedStrings.js is currently a UTF16 LE encoded file.  Though if you look at the file, it's just plain old JavaScript code.  There are some fancy quotes in some strings - search for the string "table" to find them.  And perhaps other non-ascii characters.

As this is a UTF16 file, various tools along the way to landing patches get confused.  Diff tools give up, for instance, as treat the file as binary.  Trac actually seems to display the diffs for changesets, but that's after the fact.  As it is now, it's difficult for someone reviewing a patch to see what the diffs are, and presumably very hard for someone having to merge multiple changes in.

I'd like to have a discussion of being able to treat this file somehow as an 8-byte encoded file.  Either using UTF8 encoding for the file, so folks can continue to use "fancy" characters wherever they want, or as an ASCII file where we use Unicode escaping within the JavaScript string files (eq, \u0001).

One thing that's not clear to me is what the requirements on the use of this file are, by downstream consumers.  Presumably folks like Apple use this file as input to various translation processes.

I think the worse-case scenario would be having a build step which took an 8-bit encoded version of the file and output a 16-bit version for whatever tool needs it.  

We should also do an experiment to verify that the parts of the toolchain we use that aren't friendly with the UTF16 file, are in fact friendly with a UTF8 file, if we decide that UTF8 is the format to use.  Wouldn't be much good to convert the file to UTF8 only to find we haven't fixed the real problem.
Comment 1 Timothy Hatcher 2014-01-14 11:23:48 PST
I am pretty sure we can switch to UTF-8 now. Build tools in Apple required UTF-16 until recently.
Comment 2 Radar WebKit Bug Importer 2014-03-28 14:04:02 PDT
<rdar://problem/16460975>
Comment 3 Joseph Pecoraro 2016-08-26 15:11:09 PDT
I think we are good. Lets do it!
Comment 4 Joseph Pecoraro 2016-08-26 16:22:38 PDT
Created attachment 287169 [details]
[PATCH] Proposed Fix

Next step will be just generating this as a WebInspectorUI build phase.
Comment 5 WebKit Commit Bot 2016-08-27 17:15:27 PDT
Comment on attachment 287169 [details]
[PATCH] Proposed Fix

Clearing flags on attachment: 287169

Committed r205089: <http://trac.webkit.org/changeset/205089>
Comment 6 WebKit Commit Bot 2016-08-27 17:15:52 PDT
All reviewed patches have been landed.  Closing bug.