273060 2024-04-22 01:45:18 -0700 [Win] UnicodeDecodeError on run-webkit-tests on non-English System Locale Windows 11 2024-05-16 00:06:10 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Local Build PC Other NEW P2 Trivial --- 1 kohei.asano webkit-unassigned fujii oldest_to_newest 2030152 0 kohei.asano 2024-04-22 01:45:18 -0700 I tried to run `python Tools/Scripts/run-webkit-tests --release` on Windows 11, Japanese PowerShell. I got many UnicodeDecodeError like a following message ``` UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte [16242/21282] storage/indexeddb/mozilla/key-requirements.html passed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Python311\Lib\multiprocessing\process.py", line 314, in _bootstrap self.run() File "C:\Python311\Lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\0300093582\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 308, in main typ, typ('{} (from {})'.format(str(exception), name)), traceback, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: function takes exactly 5 arguments (1 given) ``` This can be resolved by changing, system language to English via Start Menu -> Control Panel -> Region -> Admistrative -> Change system locale... 2030394 1 kohei.asano 2024-04-22 17:31:04 -0700 This error wouldn't depend on PowerShell, would cause Command Prompt as well. This is longer log, we suspect Shift_JIS from taskkill.exe is problematic. ``` [16175/21282] webgl/2.0.0/conformance2/textures/image_bitmap_from_blob/tex-2d-rgba4-rgba-unsigned_short_4_4_4_4.html pas Process Process-31: worker/30 stopping Traceback (most recent call last): File "C:\Users\\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 303, in main queue.send(_Result(value=task(None), id=task.id)) ^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 56, in __call__ return self.function(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 77, in run_shard return Worker.instance.run_tests(shard) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 325, in run_tests Worker.instance.run_test(input, shard.name) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 353, in run_test result = self._run_test_with_or_without_timeout(test_input, test_timeout_sec, stop_when_done) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 410, in _run_test_with_or_without_timeout return self._run_test_in_this_thread(test_input, stop_when_done) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 496, in _run_test_in_this_thread return self._run_single_test(self._driver, test_input, stop_when_done) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 499, in _run_single_test return single_test_runner.run_single_test( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\single_test_runner.py", line 48, in run_single_test return runner.run() ^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\single_test_runner.py", line 133, in run return self._run_compare_test() ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\single_test_runner.py", line 136, in _run_compare_test driver_output = self._driver.run_test(self._driver_input(), self._stop_when_done) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\driver.py", line 888, in run_test return self._driver.run_test(driver_input, stop_when_done) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\driver.py", line 251, in run_test out, err = self._server_process.stop(self._port.driver_stop_timeout() if stop_when_done else 0.0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\server_process.py", line 409, in stop return self._wait_for_stop(timeout_secs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\server_process.py", line 422, in _wait_for_stop self._kill() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\server_process.py", line 440, in _kill self._target_host.executive.kill_process(self._proc.pid) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\common\system\executive.py", line 199, in kill_process self.run_command(command, ignore_errors=True) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\common\system\executive.py", line 424, in run_command output = string_utils.decode(output, encoding=self._child_process_encoding()).replace('\r\n', '\n') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\string_utils.py", line 46, in decode return data.decode(encoding, errors=errors) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte ... UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte [16242/21282] storage/indexeddb/mozilla/key-requirements.html passed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Python311\Lib\multiprocessing\process.py", line 314, in _bootstrap self.run() File "C:\Python311\Lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 308, in main typ, typ('{} (from {})'.format(str(exception), name)), traceback, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: function takes exactly 5 arguments (1 given) ... [34/35] js/dom/navigator-language.html failed unexpectedly (text diff) Stopping WebSocket server ... UnicodeDecodeError raised: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte Traceback (most recent call last): File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 94, in main run_details = run(port, options, args, stderr) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 566, in run run_details = manager.run(args) ^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 470, in run self._runner.stop_servers() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 291, in stop_servers self._port.stop_websocket_server() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\base.py", line 1008, in stop_websocket_server Port._websocket_server.stop() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\http_server_base.py", line 140, in stop self._stop_running_server() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\websocket_server.py", line 178, in _stop_running_server self._wait_for_action(self._check_and_kill) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\http_server_base.py", line 199, in _wait_for_action if action(): ^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\websocket_server.py", line 200, in _check_and_kill self._executive.run_command(["taskkill.exe", "/f", "/t", "/pid", self._pid], ignore_errors=True) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\common\system\executive.py", line 424, in run_command output = string_utils.decode(output, encoding=self._child_process_encoding()).replace('\r\n', '\n') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\string_utils.py", line 46, in decode return data.decode(encoding, errors=errors) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte Stopping WebSocket server ...Failed to stop pywebsocket: pid file is missing Exception ignored in atexit callback: <function LayoutTestRunner.__init__.<locals>.<lambda> at 0x0000016806AC59E0> Traceback (most recent call last): File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 125, in <lambda> atexit.register(lambda: self.stop_servers()) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 291, in stop_servers self._port.stop_websocket_server() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\port\base.py", line 1011, in stop_websocket_server Port._websocket_secure_server.stop() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\http_server_base.py", line 140, in stop self._stop_running_server() File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\websocket_server.py", line 178, in _stop_running_server self._wait_for_action(self._check_and_kill) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\http_server_base.py", line 199, in _wait_for_action if action(): ^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\webkitpy\layout_tests\servers\websocket_server.py", line 200, in _check_and_kill self._executive.run_command(["taskkill.exe", "/f", "/t", "/pid", self._pid], ignore_errors=True) File "C:\Users\\WebKit\Tools\Scripts\webkitpy\common\system\executive.py", line 424, in run_command output = string_utils.decode(output, encoding=self._child_process_encoding()).replace('\r\n', '\n') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\\WebKit\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\string_utils.py", line 46, in decode return data.decode(encoding, errors=errors) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte ``` 2030476 2 fujii 2024-04-23 00:36:27 -0700 How about an idea of using pywin32 rather than taskkill.exe? from win32.win32process import TerminateProcess from win32.win32api import OpenProcess import win32con handle = OpenProcess(win32con.PROCESS_TERMINATE, False, 12345) TerminateProcess(handle, 0) 2035716 3 fujii 2024-05-16 00:06:10 -0700 python - subprocess: deleting child processes in Windows - Stack Overflow https://stackoverflow.com/q/1230669