Windows Python 2.7 fails to run-webkit-tests: PicklingError: Can't pickle <class 'webkitpy.common.system.executive.WrappedPopen'>: it's not found as webkitpy.common.system.executive.WrappedPopen Since r278454 (Bug 226234) PS C:\home\webkit\gc> python.exe ./Tools/Scripts/run-webkit-tests --wincairo --debug perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LC_ALL = "en_US.UTF-8", LC_MESSAGES = "en_US.UTF-8", LANG = "en_US.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C"). Using port 'wincairo-win10-wk2' Test configuration: <win10, x86_64, debug> Placing test results in C:\home\webkit\gc\WebKitBuild\Debug\bin64\layout-test-results Using Debug build Pixel tests disabled Regular timeout: 30000, slow test timeout: 150000 Command line: C:\home\webkit\gc\WebKitBuild\Debug\bin64\WebKitTestRunner.exe - Found 71450 tests; running 17972, skipping 53478. Verbose baseline search path: platform\wincairo-win10-wk2 -> platform\wincairo-win10 -> platform\wincairo-wk2 -> platform\wincairo -> platform\wk2 -> generic Baseline search path: platform\wincairo -> platform\wk2 -> generic Running 17972 tests The _NT_SYMBOL_PATH environment variable is not set. Using Microsoft Symbol Server. perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LC_ALL = "en_US.UTF-8", LC_MESSAGES = "en_US.UTF-8", LANG = "en_US.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C"). Running 16 WebKitTestRunners in parallel. Starting 16 workers ...Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main PicklingError raised: Can't pickle <class 'webkitpy.common.system.executive.WrappedPopen'>: it's not found as webkitpy.common.system.executive.WrappedPopen Traceback (most recent call last): File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 92, in main prepare(preparation_data) File "C:\Python27\lib\multiprocessing\forking.py", line 504, in prepare file, path_name, etc = imp.find_module(main_name, dirs) ImportError: No module named run-webkit-tests run_details = run(port, options, args, stderr) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 487, in run run_details = manager.run(args) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 408, in run temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 482, in _run_test_subset initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 561, in _run_tests return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 172, in run_tests ), teardown=teardown_shard, File "C:\home\webkit\gc\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 388, in __enter__ worker.start() File "C:\Python27\lib\multiprocessing\process.py", line 130, in start self._popen = Popen(self) File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__ dump(process_obj, to_child, HIGHEST_PROTOCOL) File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump ForkingPickler(file, protocol).dump(obj) File "C:\Python27\lib\pickle.py", line 224, in dump self.save(obj) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 568, in save_tuple save(element) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 396, in save_reduce save(cls) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 754, in save_global (obj, module, name)) PicklingError: Can't pickle <class 'webkitpy.common.system.executive.WrappedPopen'>: it's not found as webkitpy.common.system.executive.WrappedPopen Stopping Web Platform Test server ... PS C:\home\webkit\gc>
There is probably something on the port object we can't pickle. It's hard for me to fix this without a Windows machine, it's probably pretty straight forward, though (we had a similar problem with the helper object) I've done this in the past with something like this: def find_non_pickleable(obj): for name, value in obj.__dict__.items(): try: pickle.dumps(obj) except pickle.PicklingError: print('{} is not pickleable'.format(name)) find_non_pickleable(obj) Actually, Sam Sneddon said they might have a partial fix for this somewhere
Created attachment 430783 [details] WIP patch
One more error... Starting 16 workers ...Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main PicklingError raised: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0> Tprepare(preparation_data) raceback (most recent call last): File "C:\Python27\lib\multiprocessing\forking.py", line 504, in prepare File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 92, in main file, path_name, etc = imp.find_module(main_name, dirs) run_details = run(port, options, args, stderr) ImportError: File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 487, in run No module named run-webkit-tests run_details = manager.run(args) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 408, in run temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 482, in _run_test_subset initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 561, in _run_tests return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 172, in run_tests ), teardown=teardown_shard, File "C:\home\webkit\gc\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 388, in __enter__ worker.start() File "C:\Python27\lib\multiprocessing\process.py", line 130, in start self._popen = Popen(self) File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__ dump(process_obj, to_child, HIGHEST_PROTOCOL) File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump ForkingPickler(file, protocol).dump(obj) File "C:\Python27\lib\pickle.py", line 224, in dump self.save(obj) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 568, in save_tuple save(element) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 313, in save (t.__name__, obj)) PicklingError: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0> Stopping Web Platform Test server ... PS C:\home\webkit\gc>
find_non_pickleable reports two messages: > pretty_patch is not pickleable > RuntimeError raised: maximum recursion depth exceeded while getting the repr of an object
Created attachment 430792 [details] find_non_pickleable patch I fixed find_non_pickleable, Then I got _web_platform_test_server is not pickleable _websocket_server_temporary_directory is not pickleable _websocket_secure_server is not pickleable
Created attachment 430794 [details] WIP patch - Made _web_platform_test_server, _websocket_server_temporary_directory, and _websocket_secure_server class members
Created attachment 430795 [details] error message Still no luck. > ImportError: No module named run-webkit-tests
*** Bug 226702 has been marked as a duplicate of this bug. ***
Created attachment 430799 [details] Patch
(In reply to Fujii Hironori from comment #7) > Created attachment 430795 [details] > error message > > Still no luck. > > > ImportError: No module named run-webkit-tests That's a weird import error, since run-webkit-tests is the name of the entry point, not a module.
Comment on attachment 430799 [details] Patch Clearing flags on attachment: 430799 Committed r278624 (238609@main): <https://commits.webkit.org/238609@main>
All reviewed patches have been landed. Closing bug.
<rdar://problem/79029277>
(In reply to Fujii Hironori from comment #7) > > ImportError: No module named run-webkit-tests Filed a new ticket for it. Bug 226777 – Windows Python 2.7 fails to start run-webkit-tests due to "ImportError: No module named run-webkit-tests"