Summary: | run-webkit-tests: TaskPool reports PicklingError for Windows Python | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Fujii Hironori <Hironori.Fujii> | ||||||||||||
Component: | Tools / Tests | Assignee: | Fujii Hironori <Hironori.Fujii> | ||||||||||||
Status: | RESOLVED FIXED | ||||||||||||||
Severity: | Normal | CC: | don.olmstead, ews-watchlist, glenn, gsnedders, jbedard, webkit-bug-importer | ||||||||||||
Priority: | P2 | Keywords: | InRadar | ||||||||||||
Version: | WebKit Nightly Build | ||||||||||||||
Hardware: | Unspecified | ||||||||||||||
OS: | Unspecified | ||||||||||||||
See Also: | https://bugs.webkit.org/show_bug.cgi?id=226702 | ||||||||||||||
Attachments: |
|
Description
Fujii Hironori
2021-06-07 00:37:05 PDT
There is probably something on the port object we can't pickle. It's hard for me to fix this without a Windows machine, it's probably pretty straight forward, though (we had a similar problem with the helper object) I've done this in the past with something like this: def find_non_pickleable(obj): for name, value in obj.__dict__.items(): try: pickle.dumps(obj) except pickle.PicklingError: print('{} is not pickleable'.format(name)) find_non_pickleable(obj) Actually, Sam Sneddon said they might have a partial fix for this somewhere Created attachment 430783 [details]
WIP patch
One more error... Starting 16 workers ...Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main PicklingError raised: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0> Tprepare(preparation_data) raceback (most recent call last): File "C:\Python27\lib\multiprocessing\forking.py", line 504, in prepare File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 92, in main file, path_name, etc = imp.find_module(main_name, dirs) run_details = run(port, options, args, stderr) ImportError: File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 487, in run No module named run-webkit-tests run_details = manager.run(args) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 408, in run temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 482, in _run_test_subset initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 561, in _run_tests return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type) File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 172, in run_tests ), teardown=teardown_shard, File "C:\home\webkit\gc\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 388, in __enter__ worker.start() File "C:\Python27\lib\multiprocessing\process.py", line 130, in start self._popen = Popen(self) File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__ dump(process_obj, to_child, HIGHEST_PROTOCOL) File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump ForkingPickler(file, protocol).dump(obj) File "C:\Python27\lib\pickle.py", line 224, in dump self.save(obj) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 568, in save_tuple save(element) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "C:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "C:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "C:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "C:\Python27\lib\pickle.py", line 313, in save (t.__name__, obj)) PicklingError: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0> Stopping Web Platform Test server ... PS C:\home\webkit\gc> find_non_pickleable reports two messages: > pretty_patch is not pickleable > RuntimeError raised: maximum recursion depth exceeded while getting the repr of an object Created attachment 430792 [details]
find_non_pickleable patch
I fixed find_non_pickleable, Then I got
_web_platform_test_server is not pickleable
_websocket_server_temporary_directory is not pickleable
_websocket_secure_server is not pickleable
Created attachment 430794 [details]
WIP patch
- Made _web_platform_test_server, _websocket_server_temporary_directory, and _websocket_secure_server class members
Created attachment 430795 [details] error message Still no luck. > ImportError: No module named run-webkit-tests *** Bug 226702 has been marked as a duplicate of this bug. *** Created attachment 430799 [details]
Patch
(In reply to Fujii Hironori from comment #7) > Created attachment 430795 [details] > error message > > Still no luck. > > > ImportError: No module named run-webkit-tests That's a weird import error, since run-webkit-tests is the name of the entry point, not a module. Comment on attachment 430799 [details] Patch Clearing flags on attachment: 430799 Committed r278624 (238609@main): <https://commits.webkit.org/238609@main> All reviewed patches have been landed. Closing bug. (In reply to Fujii Hironori from comment #7) > > ImportError: No module named run-webkit-tests Filed a new ticket for it. Bug 226777 – Windows Python 2.7 fails to start run-webkit-tests due to "ImportError: No module named run-webkit-tests" |