The traceback as: $ ./check-webkit-style --help Traceback (most recent call last): File "./check-webkit-style", line 48, in <module> sys.exit(CheckWebKitStyle().main()) File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/style/main.py", line 126, in main host = Host() File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/host.py", line 49, in __init__ SystemHost.__init__(self) File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/systemhost.py", line 40, in __init__ self.user = user.User() File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/user.py", line 61, in __init__ self._platforminfo = platforminfo or PlatformInfo(sys, platform, Executive()) File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/platforminfo.py", line 56, in __init__ self.os_version = self._determine_win_version(self._win_version_tuple(sys_module)) File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/platforminfo.py", line 132, in _win_version_tuple return self._win_version_tuple_from_cmd() File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/platforminfo.py", line 136, in _win_version_tuple_from_cmd ver_output = self._executive.run_command(['cmd', '/c', 'ver']) File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/executive.py", line 402, in run_command output = output.decode(self._child_process_encoding()) File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 21: unexpected code byte
Created attachment 182481 [details] patch The version info retrieved in Chinese Windows was "Microsoft Windows [版本 6.1.7601]", "版本"('version' in English) was encoded as "0xB0 0xE6 0xB1 0xBE" and '0xB0' was not a valid utf-8 character, thus check-webkit-style running failed. Actually, we execute "cmd /c ver" then use regular expression to match version infomation, it's unnecessary to decode the output string of execution.
I thought that Executive was smart enough to use the local code page on windows? This line: File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/executive.py", line 402, in run_command output = output.decode(self._child_process_encoding()) Clearly its the wrong encoding in your case: http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/common/system/executive.py#L444 444 def _child_process_encoding(self): 445 # Win32 Python 2.x uses CreateProcessA rather than CreateProcessW 446 # to launch subprocesses, so we have to encode arguments using the 447 # current code page. 448 if sys.platform == 'win32' and sys.version < '3': 449 return 'mbcs' 450 # All other platforms use UTF-8. 451 # FIXME: Using UTF-8 on Cygwin will confuse Windows-native commands 452 # which will expect arguments to be encoded using the current code 453 # page. 454 return 'utf-8'
It looks like this article has some relevant help: http://stackoverflow.com/questions/1259084/what-encoding-code-page-is-cmd-exe-using
(In reply to comment #2) > I thought that Executive was smart enough to use the local code page on windows? I execute WebKit scripts in cygwin, sys.platform == 'cygwin' and python intergraded in cygwin did not include 'mbcs' decoder, thus it's always use 'utf-8' to decode output string. Actually GBK encoding used in my system but we could not dectect it. > This line: > File "/cygdrive/e/webkit/Tools/Scripts/webkitpy/common/system/executive.py", line 402, in run_command output = output.decode(self._child_process_encoding()) > Clearly its the wrong encoding in your case: > http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/common/system/executive.py#L444 > 444 def _child_process_encoding(self): > 445 # Win32 Python 2.x uses CreateProcessA rather than CreateProcessW > 446 # to launch subprocesses, so we have to encode arguments using the > 447 # current code page. > 448 if sys.platform == 'win32' and sys.version < '3': > 449 return 'mbcs' > 450 # All other platforms use UTF-8. > 451 # FIXME: Using UTF-8 on Cygwin will confuse Windows-native commands > 452 # which will expect arguments to be encoded using the current code > 453 # page. > 454 return 'utf-8'
(In reply to comment #3) > It looks like this article has some relevant help: > http://stackoverflow.com/questions/1259084/what-encoding-code-page-is-cmd-exe-using Thanks, I will take a look at that.
Comment on attachment 182481 [details] patch While I am not an expert, this looks fine to me.
Comment on attachment 182481 [details] patch Clearing flags on attachment: 182481 Committed r139924: <http://trac.webkit.org/changeset/139924>
All reviewed patches have been landed. Closing bug.
It is. :) It solves this specific case, but there will be many more troubles if we're not able to decode output from cmd to the right code-page on windows. We have code which should be doing this, but clearly isnt' doing it right.