REGRESSION: Some bots fail to run LayoutTests due to a WPT Server error

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Ryan Haddad]

Traceback (most recent call last):
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 89, in main
    run_details = run(port, options, args, stderr)
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 449, in run
    run_details = manager.run(args)
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 240, in run
    needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 84, in __init__
    self.start_servers()
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 201, in start_servers
    self._port.start_web_platform_test_server()
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/port/base.py", line 1015, in start_web_platform_test_server
    self._web_platform_test_server.start()
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/servers/http_server_base.py", line 98, in start
    self._pid = self._spawn_process()
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_server.py", line 164, in _spawn_process
    raise http_server_base.ServerError(error_log)
ServerError: WPT Server process exited prematurely with status code 1

https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20%28Tests%29/builds/350

[Ryan Haddad]

More log info from the bot:

CRITICAL:web-platform-test-launcher:Import of wpt serve module failed.
Please check that the file serve.py is present in the web-platform-tests folder.
Please also check that __init__.py files in the web-platform-tests/tools folder and subfolders are also present.
Traceback (most recent call last):
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", line 15, in <module>
    import tools.serve.serve as WebPlatformTestServer
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/serve/__init__.py", line 1, in <module>
    import serve
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/serve/serve.py", line 23, in <module>
    from manifest.sourcefile import read_script_metadata, js_meta_re
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/manifest/__init__.py", line 2, in <module>
    from . import manifest
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/manifest/manifest.py", line 5, in <module>
    from six import iteritems, itervalues, viewkeys
ImportError: cannot import name viewkeys

[Ryan Haddad]

I SSH'd into a few bots that *don't* exhibit this issue and it looks like they still have six as an untracked file, so that explains why they still work.

$ svn status
?       LayoutTests/imported/w3c/web-platform-tests/tools/six
?       layout-test-results.zip

We are hitting the failure on a few of our macOS and iOS bots. Two of them seemed to hit this issue after they were rebooted, so my guess is that the checkouts were cleaned up and the untracked version of six was removed.

[youenn fablet]

From the import error, it is clear that six library is missing.
On my machine, six is probably available as wpt server runs fine.
On these bots, six is probably unavailable.

The solution might be to either update these bots or add back six.
The latter might be simpler.

[youenn fablet]

Created attachment 323032 [details]
Patch

[Ryan Haddad]

(In reply to youenn fablet from comment #4)
> Created attachment 323032 [details]
> Patch
I applied this patch on one of the broken bots and it was able to start the WPT server and begin running LayoutTests.

[Build Bot]

Comment on attachment 323032 [details]
Patch

Attachment 323032 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/4783441

New failing tests:
workers/wasm-long-compile.html

[Build Bot]

Created attachment 323041 [details]
Archive of layout-test-results from ews112 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[youenn fablet]

Comment on attachment 323032 [details]
Patch

Error is unrelated.

[youenn fablet]

Comment on attachment 323032 [details]
Patch

Let's wait Monday to land this actually.

[Alexey Proskuryakov]

Any update?

[WebKit Commit Bot]

Comment on attachment 323032 [details]
Patch

Clearing flags on attachment: 323032

Committed r223064: <http://trac.webkit.org/changeset/223064>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/34893589>

[Carlos Garcia Campos]

Aha, I guess I had six installed in my system (and in all other bots). Sorry about this.