Bug 59083 - new-run-webkit-tests: Exception raised when stderr contains binary data
Summary: new-run-webkit-tests: Exception raised when stderr contains binary data
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Yuta Kitamura
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-21 02:37 PDT by Yuta Kitamura
Modified: 2011-04-22 18:49 PDT (History)
5 users (show)

See Also:


Attachments
Patch (1.45 KB, patch)
2011-04-21 02:40 PDT, Yuta Kitamura
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yuta Kitamura 2011-04-21 02:37:14 PDT
I accidentally found that NRWT gets killed when stderr output contains non-ASCII characters (0x80-0xFF).

This is because TestResultWriter writes stderr output using FileSystem.write_text_file() instead of write_binary_file(). write_text_file() implicitly converts the given data into a unicode string. In this case, input is a str containing stderr output and it gets converted with default "ascii" codec, which caused this conversion error.

I will upload a patch shortly.


Full output:

    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 74, in run
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 74, in run
      self._worker_connection.run_message_loop()
      self._worker_connection.run_message_loop()
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 192, in run_message_loop
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 192, in run_message_loop
      self._broker.run_message_loop(self._run_topic, self._client, delay_secs)
      self._broker.run_message_loop(self._run_topic, self._client, delay_secs)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 128, in run_message_loop
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 128, in run_message_loop
      self._run_loop(topic_name, client, block=True, delay_secs=delay_secs)
      self._run_loop(topic_name, client, block=True, delay_secs=delay_secs)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 142, in _run_loop
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 142, in _run_loop
      self._dispatch_message(msg, client)
      self._dispatch_message(msg, client)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 151, in _dispatch_message
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 151, in _dispatch_message
      message_handler(message.src, *optargs)
      message_handler(message.src, *optargs)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 99, in handle_test_list
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 99, in handle_test_list
      self._run_test(test_input)
      self._run_test(test_input)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 117, in _run_test
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 117, in _run_test
      result = self.run_test_with_timeout(test_input, test_timeout_sec)
      result = self.run_test_with_timeout(test_input, test_timeout_sec)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 121, in run_test_with_timeout
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 121, in run_test_with_timeout
      return self._run_test_in_this_thread(test_input)
      return self._run_test_in_this_thread(test_input)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 204, in _run_test_in_this_thread
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 204, in _run_test_in_this_thread
      return self._run_single_test(self._driver, test_input)
      return self._run_single_test(self._driver, test_input)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 208, in _run_single_test
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 208, in _run_single_test
      test_input, driver, self._name)
      test_input, driver, self._name)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 44, in run_single_test
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 44, in run_single_test
      return runner.run()
      return runner.run()
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 120, in run
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 120, in run
      return self._run_compare_test()
      return self._run_compare_test()
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 127, in _run_compare_test
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 127, in _run_compare_test
      driver_output, expected_driver_output, test_result.failures)
      driver_output, expected_driver_output, test_result.failures)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 47, in write_test_result
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 47, in write_test_result
      writer.write_stderr(driver_output.error)
      writer.write_stderr(driver_output.error)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 164, in write_stderr
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 164, in write_stderr
      fs.write_text_file(filename, error)
      fs.write_text_file(filename, error)
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 281, in write_text_file
    File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 281, in write_text_file
      f.write(contents)
      f.write(contents)
    File "/usr/lib/python2.6/codecs.py", line 686, in write
    File "/usr/lib/python2.6/codecs.py", line 686, in write
      return self.writer.write(data)
      return self.writer.write(data)
    File "/usr/lib/python2.6/codecs.py", line 351, in write
    File "/usr/lib/python2.6/codecs.py", line 351, in write
      data, consumed = self.encode(object, self.errors)
      data, consumed = self.encode(object, self.errors)

Exception raised, exiting                           
Traceback (most recent call last):
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 446, in <module>
    sys.exit(main())
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 441, in main
    return run(port_obj, options, args)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 116, in run
    num_unexpected_results = runner.run(result_summary)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner.py", line 652, in run
    self._run_tests(self._test_files_list, result_summary))
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner2.py", line 164, in _run_tests
    manager_connection.run_message_loop(delay_secs=1.0)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 192, in run_message_loop
    self._broker.run_message_loop(self._run_topic, self._client, delay_secs)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 128, in run_message_loop
    self._run_loop(topic_name, client, block=True, delay_secs=delay_secs)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 142, in _run_loop
    self._dispatch_message(msg, client)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 151, in _dispatch_message
    message_handler(message.src, *optargs)
  File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner2.py", line 216, in handle_exception
    raise exception_type, exception_value, exception_traceback
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 146: ordinal not in range(128)
Comment 1 Yuta Kitamura 2011-04-21 02:40:31 PDT
Created attachment 90505 [details]
Patch
Comment 2 Ojan Vafai 2011-04-21 08:37:43 PDT
Comment on attachment 90505 [details]
Patch

Seems fine.
Comment 3 WebKit Commit Bot 2011-04-21 17:11:35 PDT
Comment on attachment 90505 [details]
Patch

Rejecting attachment 90505 [details] from commit-queue.

Failed to run "['./Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=cr-jail-3', 'apply-..." exit_code: 1

Last 500 characters of output:
autoinstalled/mechanize/_urllib2_fork.py", line 332, in _call_chain
    result = func(*args)
  File "/mnt/git/webkit-commit-queue/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 1170, in https_open
    return self.do_open(conn_factory, req)
  File "/mnt/git/webkit-commit-queue/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 1118, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 60] Operation timed out>

Full output: http://queues.webkit.org/results/8498155
Comment 4 Yuta Kitamura 2011-04-22 15:00:00 PDT
Um... bot flakiness? Let me try again.
Comment 5 Eric Seidel (no email) 2011-04-22 15:01:20 PDT
yeah, not sure why the server timed out like that.
Comment 6 WebKit Commit Bot 2011-04-22 18:49:08 PDT
Comment on attachment 90505 [details]
Patch

Clearing flags on attachment: 90505

Committed r84722: <http://trac.webkit.org/changeset/84722>
Comment 7 WebKit Commit Bot 2011-04-22 18:49:13 PDT
All reviewed patches have been landed.  Closing bug.