Bug 190258 – [GTK] build fails with python 3 if LANG and LC_TYPE are unset

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Mike Gorse]

Building the gtk port still fails using Python 3 if LANG or LC_TYPE do not specify UTF-8. I'm seeing tracebacks such as the following:
Traceback (most recent call last):
  File "/home/mgorse/src/WebKit/Source/JavaScriptCore/Scripts/generateIntlCanonicalizeLanguage.py", line 197, in <module>
    registry.parse(registryFile)
  File "/home/mgorse/src/WebKit/Source/JavaScriptCore/Scripts/generateIntlCanonicalizeLanguage.py", line 63, in parse
    for line in file:
  File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 866: ordinal not in range(128)

[Mike Gorse]

Created attachment 351542 [details]
Proposed patch.

Explicitly specify UTF-8 encoding for Python 3 in some places, rather than defaulting to the encoding specified in the environment.

[Mark Lam]

Comment on attachment 351542 [details]
Proposed patch.

LGTM but let's wait for the EWS bots to be green before committing.

[Konstantin Tokarev]

Comment on attachment 351542 [details]
Proposed patch.

View in context: https://bugs.webkit.org/attachment.cgi?id=351542&action=review

> Source/JavaScriptCore/Scripts/cssmin.py:51
> +        sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='UTF-8')

These lines duplicate code above

[Mike Gorse]

(In reply to Konstantin Tokarev from comment #3)
> Comment on attachment 351542 [details]
> Proposed patch.
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=351542&action=review
> 
> > Source/JavaScriptCore/Scripts/cssmin.py:51
> > +        sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='UTF-8')
> 
> These lines duplicate code above

I think you're referring to this line:
         sys.stdin = io.TextIOWrapper(sys.stdin.buffer, encoding='UTF-8')
My patch adds similar handling for sys.stdout.

[Konstantin Tokarev]

Comment on attachment 351542 [details]
Proposed patch.

View in context: https://bugs.webkit.org/attachment.cgi?id=351542&action=review

>>> Source/JavaScriptCore/Scripts/cssmin.py:51
>>> +        sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='UTF-8')
>> 
>> These lines duplicate code above
> 
> I think you're referring to this line:
>          sys.stdin = io.TextIOWrapper(sys.stdin.buffer, encoding='UTF-8')
> My patch adds similar handling for sys.stdout.

Oh, sorry. Still I think it would be better to avoid duplication of 'sys.version_info[0] >= 3' and 'import io' to prevent misreads

[Mike Gorse]

Created attachment 351643 [details]
Updated patch.

Updated cssmin.py to remove duplicated if check and import.

[Michael Catanzaro]

If you keep the previous reviewed by line when you upload it (either by adding it manually or by applying it with 'webkit-patch apply-from-bug') then any committer would be able to commit it without waiting for Konstantin again.

[WebKit Commit Bot]

Comment on attachment 351643 [details]
Updated patch.

Clearing flags on attachment: 351643

Committed r236870: <https://trac.webkit.org/changeset/236870>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/45038771>