Bug 135502

Summary: [GTK] generate-gtkdoc crashes when generating HTML due to encoding issues
Product: WebKit Reporter: Carlos Garcia Campos <cgarcia>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: mrobinson, pnormand
Priority: P2 Keywords: Gtk
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
This should work for everybody mrobinson: review+

Description Carlos Garcia Campos 2014-08-01 04:12:24 PDT
Generating webkit2gtk documentation...
Copying template files to output directory...
Copying HTML files to output directory...
Running gtkdoc-scan
Running gtkdoc-scangobj
Running gtkdoc-mktmpl
Running gtkdoc-mkdb
Running gtkdoc-mkhtml
Running gtkdoc-fixxref
Traceback (most recent call last):
  File "../../Tools/gtk/generate-gtkdoc", line 202, in <module>
    saw_warnings = generate_documentation_for_config(common.build_path('gtkdoc-webkit2gtk.cfg'))
  File "../../Tools/gtk/generate-gtkdoc", line 152, in generate_documentation_for_config
    return generate_doc(generator, arguments.skip_html)
  File "../../Tools/gtk/generate-gtkdoc", line 133, in generate_doc
    generator.generate(not skip_html)
  File "../../Tools/gtk/gtkdoc.py", line 151, in generate
    self._run_gtkdoc_fixxref()
  File "../../Tools/gtk/gtkdoc.py", line 379, in _run_gtkdoc_fixxref
    self._run_command(args, cwd=self.output_dir, ignore_warnings=True)
  File "../../Tools/gtk/gtkdoc.py", line 198, in _run_command
    sys.stdout.write(stdout.encode("utf-8"))
  File "/usr/lib/python2.7/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 883: ordinal not in range(128)
Comment 1 Carlos Garcia Campos 2014-08-01 04:15:44 PDT
It seems it's trying to convert ascii text to utf8, but the text is not ascii, but utf8 already. This line is causing the problem:

html/WebKitContextMenuItem.html:200: warning: no link for: 'GtkAction-activate' -> (<span class="type">“activate”</span>).

Because of the quotation marks in “activate”. The line crashing is:

sys.stdout.write(stdout.encode("utf-8"))

and it works if we just remove the encode

sys.stdout.write(stdout)
Comment 2 Martin Robinson 2014-08-01 11:28:07 PDT
I guess the output should be decoded as utf-8?
Comment 3 Carlos Garcia Campos 2014-08-05 23:13:23 PDT
(In reply to comment #2)
> I guess the output should be decoded as utf-8?

I think we should simply remove all .encode('utf-8')
Comment 4 Carlos Garcia Campos 2014-08-14 01:43:48 PDT
Created attachment 236582 [details]
Patch
Comment 5 Philippe Normand 2014-08-14 01:52:25 PDT
I think Martin is right, decode should be used.
Comment 6 Carlos Garcia Campos 2014-08-14 02:07:19 PDT
(In reply to comment #5)
> I think Martin is right, decode should be used.

I don't see why we need to mess up with stdout/stderr encoding, what we get from the command is what we send to stderr/stdout
Comment 7 Carlos Garcia Campos 2014-09-25 01:39:48 PDT
Created attachment 238649 [details]
This should work for everybody
Comment 8 Carlos Garcia Campos 2014-10-04 00:35:58 PDT
Committed r174309: <http://trac.webkit.org/changeset/174309>