Bug 38505 – new-run-webkit-tests hung on Chromium Mac bot writing to files

• Status: RESOLVED
• Resolution: WORKSFORME
• Component: Tools / Tests

[Eric Seidel (no email)]

new-run-webkit-tests hung on Chromium Mac bot writing to files

http://build.chromium.org/buildbot/waterfall.fyi/builders/Webkit%20Mac%20(webkit.org)/builds/19185/steps/webkit_tests/logs/stdio
Copy of the log attached.

The interesting bits:

# Thread: -1341116416
File: "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py", line 460, in __bootstrap
  self.run()
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 258, in run
  self._run(test_runner=None, result_summary=None)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 330, in _run
  result = self._run_test(test_info)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 439, in _run_test
  output, error)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 107, in process_output
  configuration)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/test_types/text_diff.py", line 109, in compare_output
  print_text_diffs=True)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/test_types/test_type_base.py", line 209, in write_output_files
  pretty_patch = port.pretty_patch_text(diff_filename)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/port/base.py", line 603, in pretty_patch_text
  return self._executive.run_command(command, decode_output=False)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/common/system/executive.py", line 250, in run_command
  output = process.communicate(string_to_communicate)[0]
File: "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/subprocess.py", line 661, in communicate
  stdout = self.stdout.read()

# Thread: -1341648896
File: "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py", line 460, in __bootstrap
  self.run()
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 258, in run
  self._run(test_runner=None, result_summary=None)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 330, in _run
  result = self._run_test(test_info)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 439, in _run_test
  output, error)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py", line 107, in process_output
  configuration)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/test_types/text_diff.py", line 109, in compare_output
  print_text_diffs=True)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/test_types/test_type_base.py", line 201, in write_output_files
  self._write_into_file_at_path(diff_filename, diff, encoding=None)
File: "/b/slave/webkit-rel-mac-webkit-org/build/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/test_types/test_type_base.py", line 160, in _write_into_file_at_path
  file.write(contents)


One thread is waiting for a response from pretypatch.  The other thread is hung writing the results from running "diff" at:
        self._write_into_file_at_path(diff_filename, diff, encoding=None)

_write_file_into_path calls "codecs.open(..., encoding=encoding)".  enoding=None just calls through to __builtin__.open(), so this should be no different from writing any other file.

It's certainly possible that prettypatch got wedged and never print anything.  Not sure.

This also could just be a symptom of whatever python bug is causing bug 38300.

[Eric Seidel (no email)]

Created attachment 54990 [details]
failure log from chromium bot

Contents of http://build.chromium.org/buildbot/waterfall.fyi/builders/Webkit%20Mac%20(webkit.org)/builds/19185/steps/webkit_tests/logs/stdio in case it gets purged.

[Eric Seidel (no email)]

It appears that calling PretyPatch.pretty_diff("") will hang.  So I suspect that explains one of the threads hanging.  Should be an easy fix.

I'm not sure how you'd ever see file.write() hang though.

[Dirk Pranke]

From a fairly quick look over the past  few days on the Chromium bots, several bots (WebKit Mac, WebKit Mac (dbg)(3) at least) are hanging with several different sets of symptoms. It looks like most of those hangs are probably not related to pretty patch, since it looks like pretty patch may not be available. I will try to dig up some representative stack traces.

[Dirk Pranke]

I'm assuming this is fixed by the rewrite or something else since we presumably no longer have this constantly happening. Please reopen if it's still relevant.