Bug 59083

Summary:

new-run-webkit-tests: Exception raised when stderr contains binary data

Product:

WebKit

Reporter:

Yuta Kitamura <yutak>

Component:

Tools / Tests

Assignee:

Yuta Kitamura <yutak>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

commit-queue, dpranke, eric, ojan, tony

Priority:

Version:

528+ (Nightly build)

Hardware:

All

OS:

All

Attachments:

Description	Flags
Patch	none

Yuta Kitamura

Reported 2011-04-21 02:37:14 PDT

I accidentally found that NRWT gets killed when stderr output contains non-ASCII characters (0x80-0xFF). This is because TestResultWriter writes stderr output using FileSystem.write_text_file() instead of write_binary_file(). write_text_file() implicitly converts the given data into a unicode string. In this case, input is a str containing stderr output and it gets converted with default "ascii" codec, which caused this conversion error. I will upload a patch shortly. Full output: File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 74, in run File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 74, in run self._worker_connection.run_message_loop() self._worker_connection.run_message_loop() File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 192, in run_message_loop File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 192, in run_message_loop self._broker.run_message_loop(self._run_topic, self._client, delay_secs) self._broker.run_message_loop(self._run_topic, self._client, delay_secs) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 128, in run_message_loop File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 128, in run_message_loop self._run_loop(topic_name, client, block=True, delay_secs=delay_secs) self._run_loop(topic_name, client, block=True, delay_secs=delay_secs) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 142, in _run_loop File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 142, in _run_loop self._dispatch_message(msg, client) self._dispatch_message(msg, client) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 151, in _dispatch_message File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 151, in _dispatch_message message_handler(message.src, *optargs) message_handler(message.src, *optargs) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 99, in handle_test_list File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 99, in handle_test_list self._run_test(test_input) self._run_test(test_input) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 117, in _run_test File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker.py", line 117, in _run_test result = self.run_test_with_timeout(test_input, test_timeout_sec) result = self.run_test_with_timeout(test_input, test_timeout_sec) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 121, in run_test_with_timeout File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 121, in run_test_with_timeout return self._run_test_in_this_thread(test_input) return self._run_test_in_this_thread(test_input) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 204, in _run_test_in_this_thread File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 204, in _run_test_in_this_thread return self._run_single_test(self._driver, test_input) return self._run_single_test(self._driver, test_input) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 208, in _run_single_test File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py", line 208, in _run_single_test test_input, driver, self._name) test_input, driver, self._name) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 44, in run_single_test File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 44, in run_single_test return runner.run() return runner.run() File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 120, in run File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 120, in run return self._run_compare_test() return self._run_compare_test() File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 127, in _run_compare_test File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/single_test_runner.py", line 127, in _run_compare_test driver_output, expected_driver_output, test_result.failures) driver_output, expected_driver_output, test_result.failures) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 47, in write_test_result File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 47, in write_test_result writer.write_stderr(driver_output.error) writer.write_stderr(driver_output.error) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 164, in write_stderr File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py", line 164, in write_stderr fs.write_text_file(filename, error) fs.write_text_file(filename, error) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 281, in write_text_file File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 281, in write_text_file f.write(contents) f.write(contents) File "/usr/lib/python2.6/codecs.py", line 686, in write File "/usr/lib/python2.6/codecs.py", line 686, in write return self.writer.write(data) return self.writer.write(data) File "/usr/lib/python2.6/codecs.py", line 351, in write File "/usr/lib/python2.6/codecs.py", line 351, in write data, consumed = self.encode(object, self.errors) data, consumed = self.encode(object, self.errors) Exception raised, exiting Traceback (most recent call last): File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 446, in <module> sys.exit(main()) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 441, in main return run(port_obj, options, args) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 116, in run num_unexpected_results = runner.run(result_summary) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner.py", line 652, in run self._run_tests(self._test_files_list, result_summary)) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner2.py", line 164, in _run_tests manager_connection.run_message_loop(delay_secs=1.0) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 192, in run_message_loop self._broker.run_message_loop(self._run_topic, self._client, delay_secs) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 128, in run_message_loop self._run_loop(topic_name, client, block=True, delay_secs=delay_secs) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 142, in _run_loop self._dispatch_message(msg, client) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/message_broker2.py", line 151, in _dispatch_message message_handler(message.src, *optargs) File "/usr/local/google/home/yutak/build/Chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner2.py", line 216, in handle_exception raise exception_type, exception_value, exception_traceback UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 146: ordinal not in range(128)

Attachments
Patch (1.45 KB, patch) 2011-04-21 02:40 PDT, Yuta Kitamura	no flags	Details Formatted Diff Diff
View All Add attachment proposed patch, testcase, etc.

Yuta Kitamura

Comment 1 2011-04-21 02:40:31 PDT

Created attachment 90505 [details] Patch

Ojan Vafai

Comment 2 2011-04-21 08:37:43 PDT

Comment on attachment 90505 [details] Patch Seems fine.

WebKit Commit Bot

Comment 3 2011-04-21 17:11:35 PDT

Comment on attachment 90505 [details] Patch Rejecting attachment 90505 [details] from commit-queue. Failed to run "['./Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=cr-jail-3', 'apply-..." exit_code: 1 Last 500 characters of output: autoinstalled/mechanize/_urllib2_fork.py", line 332, in _call_chain result = func(*args) File "/mnt/git/webkit-commit-queue/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 1170, in https_open return self.do_open(conn_factory, req) File "/mnt/git/webkit-commit-queue/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 1118, in do_open raise URLError(err) urllib2.URLError: <urlopen error [Errno 60] Operation timed out> Full output: http://queues.webkit.org/results/8498155

Yuta Kitamura

Comment 4 2011-04-22 15:00:00 PDT

Um... bot flakiness? Let me try again.

Eric Seidel (no email)

Comment 5 2011-04-22 15:01:20 PDT

yeah, not sure why the server timed out like that.

WebKit Commit Bot

Comment 6 2011-04-22 18:49:08 PDT

Comment on attachment 90505 [details] Patch Clearing flags on attachment: 90505 Committed r84722: <http://trac.webkit.org/changeset/84722>

WebKit Commit Bot

Comment 7 2011-04-22 18:49:13 PDT

All reviewed patches have been landed. Closing bug.

Note You need to log in before you can comment on or make changes to this bug.