| Summary: | run-webkit-tests: TaskPool reports PicklingError for Windows Python | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Fujii Hironori <Hironori.Fujii> | ||||||||||||
| Component: | Tools / Tests | Assignee: | Fujii Hironori <Hironori.Fujii> | ||||||||||||
| Status: | RESOLVED FIXED | ||||||||||||||
| Severity: | Normal | CC: | don.olmstead, ews-watchlist, glenn, gsnedders, jbedard, webkit-bug-importer | ||||||||||||
| Priority: | P2 | Keywords: | InRadar | ||||||||||||
| Version: | WebKit Nightly Build | ||||||||||||||
| Hardware: | Unspecified | ||||||||||||||
| OS: | Unspecified | ||||||||||||||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=226702 | ||||||||||||||
| Attachments: |
|
||||||||||||||
|
Description
Fujii Hironori
2021-06-07 00:37:05 PDT
There is probably something on the port object we can't pickle. It's hard for me to fix this without a Windows machine, it's probably pretty straight forward, though (we had a similar problem with the helper object)
I've done this in the past with something like this:
def find_non_pickleable(obj):
for name, value in obj.__dict__.items():
try:
pickle.dumps(obj)
except pickle.PicklingError:
print('{} is not pickleable'.format(name))
find_non_pickleable(obj)
Actually, Sam Sneddon said they might have a partial fix for this somewhere
Created attachment 430783 [details]
WIP patch
One more error...
Starting 16 workers ...Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main
PicklingError raised: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0>
Tprepare(preparation_data)
raceback (most recent call last):
File "C:\Python27\lib\multiprocessing\forking.py", line 504, in prepare
File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 92, in main
file, path_name, etc = imp.find_module(main_name, dirs)
run_details = run(port, options, args, stderr)
ImportError: File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 487, in run
No module named run-webkit-tests
run_details = manager.run(args)
File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 408, in run
temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type)
File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 482, in _run_test_subset
initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type)
File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 561, in _run_tests
return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type)
File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 172, in run_tests
), teardown=teardown_shard,
File "C:\home\webkit\gc\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 388, in __enter__
worker.start()
File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
self._popen = Popen(self)
File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 568, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 313, in save
(t.__name__, obj))
PicklingError: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0>
Stopping Web Platform Test server ...
PS C:\home\webkit\gc>
find_non_pickleable reports two messages: > pretty_patch is not pickleable > RuntimeError raised: maximum recursion depth exceeded while getting the repr of an object Created attachment 430792 [details]
find_non_pickleable patch
I fixed find_non_pickleable, Then I got
_web_platform_test_server is not pickleable
_websocket_server_temporary_directory is not pickleable
_websocket_secure_server is not pickleable
Created attachment 430794 [details]
WIP patch
- Made _web_platform_test_server, _websocket_server_temporary_directory, and _websocket_secure_server class members
Created attachment 430795 [details] error message Still no luck. > ImportError: No module named run-webkit-tests *** Bug 226702 has been marked as a duplicate of this bug. *** Created attachment 430799 [details]
Patch
(In reply to Fujii Hironori from comment #7) > Created attachment 430795 [details] > error message > > Still no luck. > > > ImportError: No module named run-webkit-tests That's a weird import error, since run-webkit-tests is the name of the entry point, not a module. Comment on attachment 430799 [details] Patch Clearing flags on attachment: 430799 Committed r278624 (238609@main): <https://commits.webkit.org/238609@main> All reviewed patches have been landed. Closing bug. (In reply to Fujii Hironori from comment #7) > > ImportError: No module named run-webkit-tests Filed a new ticket for it. Bug 226777 – Windows Python 2.7 fails to start run-webkit-tests due to "ImportError: No module named run-webkit-tests" |