Bug 178017

Summary: REGRESSION: Some bots fail to run LayoutTests due to a WPT Server error
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: Tools / TestsAssignee: youenn fablet <youennf>
Severity: Normal CC: ap, buildbot, cdumez, cgarcia, commit-queue, jlewis3, lforschler, sam, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=177351
Description Flags
Archive of layout-test-results from ews112 for mac-elcapitan none

Description Ryan Haddad 2017-10-06 10:29:53 PDT
Traceback (most recent call last):
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 89, in main
    run_details = run(port, options, args, stderr)
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 449, in run
    run_details = manager.run(args)
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 240, in run
    needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 84, in __init__
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 201, in start_servers
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/port/base.py", line 1015, in start_web_platform_test_server
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/servers/http_server_base.py", line 98, in start
    self._pid = self._spawn_process()
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_server.py", line 164, in _spawn_process
    raise http_server_base.ServerError(error_log)
ServerError: WPT Server process exited prematurely with status code 1

Comment 1 Ryan Haddad 2017-10-06 10:30:52 PDT
More log info from the bot:

CRITICAL:web-platform-test-launcher:Import of wpt serve module failed.
Please check that the file serve.py is present in the web-platform-tests folder.
Please also check that __init__.py files in the web-platform-tests/tools folder and subfolders are also present.
Traceback (most recent call last):
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", line 15, in <module>
    import tools.serve.serve as WebPlatformTestServer
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/serve/__init__.py", line 1, in <module>
    import serve
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/serve/serve.py", line 23, in <module>
    from manifest.sourcefile import read_script_metadata, js_meta_re
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/manifest/__init__.py", line 2, in <module>
    from . import manifest
  File "/Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/LayoutTests/imported/w3c/web-platform-tests/tools/manifest/manifest.py", line 5, in <module>
    from six import iteritems, itervalues, viewkeys
ImportError: cannot import name viewkeys
Comment 2 Ryan Haddad 2017-10-06 10:39:40 PDT
I SSH'd into a few bots that *don't* exhibit this issue and it looks like they still have six as an untracked file, so that explains why they still work.

$ svn status
?       LayoutTests/imported/w3c/web-platform-tests/tools/six
?       layout-test-results.zip

We are hitting the failure on a few of our macOS and iOS bots. Two of them seemed to hit this issue after they were rebooted, so my guess is that the checkouts were cleaned up and the untracked version of six was removed.
Comment 3 youenn fablet 2017-10-06 11:23:50 PDT
From the import error, it is clear that six library is missing.
On my machine, six is probably available as wpt server runs fine.
On these bots, six is probably unavailable.

The solution might be to either update these bots or add back six.
The latter might be simpler.
Comment 4 youenn fablet 2017-10-06 11:28:04 PDT
Created attachment 323032 [details]
Comment 5 Ryan Haddad 2017-10-06 12:01:32 PDT
(In reply to youenn fablet from comment #4)
> Created attachment 323032 [details]
> Patch
I applied this patch on one of the broken bots and it was able to start the WPT server and begin running LayoutTests.
Comment 6 Build Bot 2017-10-06 13:46:04 PDT
Comment on attachment 323032 [details]

Attachment 323032 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/4783441

New failing tests:
Comment 7 Build Bot 2017-10-06 13:46:05 PDT
Created attachment 323041 [details]
Archive of layout-test-results from ews112 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 8 youenn fablet 2017-10-06 13:46:40 PDT
Comment on attachment 323032 [details]

Error is unrelated.
Comment 9 youenn fablet 2017-10-06 21:26:16 PDT
Comment on attachment 323032 [details]

Let's wait Monday to land this actually.
Comment 10 Alexey Proskuryakov 2017-10-09 12:59:38 PDT
Any update?
Comment 11 WebKit Commit Bot 2017-10-09 13:23:36 PDT
Comment on attachment 323032 [details]

Clearing flags on attachment: 323032

Committed r223064: <http://trac.webkit.org/changeset/223064>
Comment 12 WebKit Commit Bot 2017-10-09 13:23:37 PDT
All reviewed patches have been landed.  Closing bug.
Comment 13 Radar WebKit Bug Importer 2017-10-09 13:24:32 PDT
Comment 14 Carlos Garcia Campos 2017-10-10 03:45:41 PDT
Aha, I guess I had six installed in my system (and in all other bots). Sorry about this.