Bug 226710 – run-webkit-tests: TaskPool reports PicklingError for Windows Python

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Fujii Hironori]

Windows Python 2.7 fails to run-webkit-tests: PicklingError: Can't pickle <class 'webkitpy.common.system.executive.WrappedPopen'>: it's not found as webkitpy.common.system.executive.WrappedPopen

Since r278454 (Bug 226234)

PS C:\home\webkit\gc> python.exe ./Tools/Scripts/run-webkit-tests --wincairo --debug
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LC_ALL = "en_US.UTF-8",
        LC_MESSAGES = "en_US.UTF-8",
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Using port 'wincairo-win10-wk2'
Test configuration: <win10, x86_64, debug>
Placing test results in C:\home\webkit\gc\WebKitBuild\Debug\bin64\layout-test-results
Using Debug build
Pixel tests disabled
Regular timeout: 30000, slow test timeout: 150000
Command line: C:\home\webkit\gc\WebKitBuild\Debug\bin64\WebKitTestRunner.exe -

Found 71450 tests; running 17972, skipping 53478.

Verbose baseline search path: platform\wincairo-win10-wk2 -> platform\wincairo-win10 -> platform\wincairo-wk2 -> platform\wincairo -> platform\wk2 -> generic

Baseline search path: platform\wincairo -> platform\wk2 -> generic

Running 17972 tests

The _NT_SYMBOL_PATH environment variable is not set. Using Microsoft Symbol Server.
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LC_ALL = "en_US.UTF-8",
        LC_MESSAGES = "en_US.UTF-8",
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Running 16 WebKitTestRunners in parallel.

Starting 16 workers ...Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main

PicklingError raised: Can't pickle <class 'webkitpy.common.system.executive.WrappedPopen'>: it's not found as webkitpy.common.system.executive.WrappedPopen
Traceback (most recent call last):
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 92, in main
    prepare(preparation_data)
  File "C:\Python27\lib\multiprocessing\forking.py", line 504, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named run-webkit-tests    run_details = run(port, options, args, stderr)

 File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 487, in run
    run_details = manager.run(args)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 408, in run
    temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 482, in _run_test_subset
    initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 561, in _run_tests
    return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 172, in run_tests
    ), teardown=teardown_shard,
  File "C:\home\webkit\gc\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 388, in __enter__
    worker.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 568, in save_tuple
    save(element)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 396, in save_reduce
    save(cls)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 754, in save_global
    (obj, module, name))
PicklingError: Can't pickle <class 'webkitpy.common.system.executive.WrappedPopen'>: it's not found as webkitpy.common.system.executive.WrappedPopen
Stopping Web Platform Test server ...
PS C:\home\webkit\gc>

[Jonathan Bedard]

There is probably something on the port object we can't pickle. It's hard for me to fix this without a Windows machine, it's probably pretty straight forward, though (we had a similar problem with the helper object)

I've done this in the past with something like this:

def find_non_pickleable(obj):
    for name, value in obj.__dict__.items():
        try:
            pickle.dumps(obj)
        except pickle.PicklingError:
            print('{} is not pickleable'.format(name))
            find_non_pickleable(obj)

Actually, Sam Sneddon said they might have a partial fix for this somewhere

[Fujii Hironori]

Created attachment 430783 [details]
WIP patch

[Fujii Hironori]

One more error...

Starting 16 workers ...Traceback (most recent call last):
  File "<string>", line 1, in <module>

  File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main
PicklingError raised: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0>
    Tprepare(preparation_data)
raceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\forking.py", line 504, in prepare
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 92, in main
    file, path_name, etc = imp.find_module(main_name, dirs)
    run_details = run(port, options, args, stderr)
ImportError:   File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 487, in run
No module named run-webkit-tests
    run_details = manager.run(args)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 408, in run
    temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 482, in _run_test_subset
    initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 561, in _run_tests
    return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type)
  File "C:\home\webkit\gc\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 172, in run_tests
    ), teardown=teardown_shard,
  File "C:\home\webkit\gc\Tools\Scripts\libraries\webkitcorepy\webkitcorepy\task_pool.py", line 388, in __enter__
    worker.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 568, in save_tuple
    save(element)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 313, in save
    (t.__name__, obj))
PicklingError: Can't pickle '_subprocess_handle' object: <_subprocess_handle object at 0x00000000080AA2B0>
Stopping Web Platform Test server ...
PS C:\home\webkit\gc>

[Fujii Hironori]

find_non_pickleable reports two messages:

> pretty_patch is not pickleable

> RuntimeError raised: maximum recursion depth exceeded while getting the repr of an object

[Fujii Hironori]

Created attachment 430792 [details]
find_non_pickleable patch

I fixed find_non_pickleable, Then I got

_web_platform_test_server is not pickleable
_websocket_server_temporary_directory is not pickleable
_websocket_secure_server is not pickleable

[Fujii Hironori]

Created attachment 430794 [details]
WIP patch

- Made _web_platform_test_server, _websocket_server_temporary_directory, and _websocket_secure_server class members

[Fujii Hironori]

Created attachment 430795 [details]
error message

Still no luck.

> ImportError: No module named run-webkit-tests

[Fujii Hironori]

*** Bug 226702 has been marked as a duplicate of this bug. ***

[Fujii Hironori]

Created attachment 430799 [details]
Patch

[Jonathan Bedard]

(In reply to Fujii Hironori from comment #7)
> Created attachment 430795 [details]
> error message
> 
> Still no luck.
> 
> > ImportError: No module named run-webkit-tests

That's a weird import error, since run-webkit-tests is the name of the entry point, not a module.

[Fujii Hironori]

Comment on attachment 430799 [details]
Patch

Clearing flags on attachment: 430799

Committed r278624 (238609@main): <https://commits.webkit.org/238609@main>

[Fujii Hironori]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/79029277>

[Fujii Hironori]

(In reply to Fujii Hironori from comment #7)
> > ImportError: No module named run-webkit-tests

Filed a new ticket for it.
  Bug 226777 – Windows Python 2.7 fails to start run-webkit-tests due to "ImportError: No module named run-webkit-tests"