Bug 135502 – [GTK] generate-gtkdoc crashes when generating HTML due to encoding issues

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Carlos Garcia Campos]

Generating webkit2gtk documentation...
Copying template files to output directory...
Copying HTML files to output directory...
Running gtkdoc-scan
Running gtkdoc-scangobj
Running gtkdoc-mktmpl
Running gtkdoc-mkdb
Running gtkdoc-mkhtml
Running gtkdoc-fixxref
Traceback (most recent call last):
  File "../../Tools/gtk/generate-gtkdoc", line 202, in <module>
    saw_warnings = generate_documentation_for_config(common.build_path('gtkdoc-webkit2gtk.cfg'))
  File "../../Tools/gtk/generate-gtkdoc", line 152, in generate_documentation_for_config
    return generate_doc(generator, arguments.skip_html)
  File "../../Tools/gtk/generate-gtkdoc", line 133, in generate_doc
    generator.generate(not skip_html)
  File "../../Tools/gtk/gtkdoc.py", line 151, in generate
    self._run_gtkdoc_fixxref()
  File "../../Tools/gtk/gtkdoc.py", line 379, in _run_gtkdoc_fixxref
    self._run_command(args, cwd=self.output_dir, ignore_warnings=True)
  File "../../Tools/gtk/gtkdoc.py", line 198, in _run_command
    sys.stdout.write(stdout.encode("utf-8"))
  File "/usr/lib/python2.7/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 883: ordinal not in range(128)

[Carlos Garcia Campos]

It seems it's trying to convert ascii text to utf8, but the text is not ascii, but utf8 already. This line is causing the problem:

html/WebKitContextMenuItem.html:200: warning: no link for: 'GtkAction-activate' -> (<span class="type">“activate”</span>).

Because of the quotation marks in “activate”. The line crashing is:

sys.stdout.write(stdout.encode("utf-8"))

and it works if we just remove the encode

sys.stdout.write(stdout)

[Martin Robinson]

I guess the output should be decoded as utf-8?

[Carlos Garcia Campos]

(In reply to comment #2)
> I guess the output should be decoded as utf-8?

I think we should simply remove all .encode('utf-8')

[Carlos Garcia Campos]

Created attachment 236582 [details]
Patch

[Philippe Normand]

I think Martin is right, decode should be used.

[Carlos Garcia Campos]

(In reply to comment #5)
> I think Martin is right, decode should be used.

I don't see why we need to mess up with stdout/stderr encoding, what we get from the command is what we send to stderr/stdout

[Carlos Garcia Campos]

Created attachment 238649 [details]
This should work for everybody

[Carlos Garcia Campos]

Committed r174309: <http://trac.webkit.org/changeset/174309>