Bug 81967 – lots of cookied crashes when running nrwt -2 on mac lion, no webprocess crash logs

• Status: NEW
• Component: Tools / Tests

[Dirk Pranke]

this seems to confuse WebKitTestRunner, insofar as it reports that the WebProcess is crashing but we don't get a stack trace for it. Per ap@ on IRC, this is to be expected since WebProcess is calling exit(); maybe there should be a mode to abort and generate a stack instead if that would be useful?

At any rate, it seems like since cookied is a system process, it crashing is a bad thing ... Dunno who to cc on this, so hopefully someone can point this bug in the right direction.

[Dirk Pranke]

Seen on my 10.7.3 desktop using a release build of r111765.

[Alexey Proskuryakov]

> this seems to confuse WebKitTestRunner

I actually don't think that a cookied crash would cause this. These issues seem unrelated.

[Dirk Pranke]

It's possible that WTR is functioning correctly, and reporting WebProcess crashes as well that just don't show up.

It probably would be nice to figure out whether the WebProcess exits cleanly or not and whether we should look for a crash log, since we could spend a long time blocked waiting for one to show up (up to 80 seconds on an 8-core hyperthreaded Mac Pro) unnecessarily.

I’ve run the tests on my Lion machine and got a few crashes in results.html that linked to a file that said “no crash log found for WebProcess:<pid>”, but looking in ~/Library/Logs/DiagnosticMessages I found a crash report corresponding to the pid. In all cases I checked, the Web process did not exist cleanly. It crashed, and a crash report was present.

[Dirk Pranke]

It's also possible that we're not waiting long enough for the crash logs ... can you try running again, and modify Tools/Scripts/webkitpy/layout_tests/port/mac.py:210 to:

      crash_log = 'no crash log found for %s:%d (now >= deadline: %s)' % (name, pid, str(now >= deadline))            

and let me know if that is printing (now >= deadline: True)  ?

(I will try this as well)

[Dirk Pranke]

Are we still seeing issues here?

[Brian Weinstein]

It looks like we are still seeing this.

http://build.webkit.org/results/Apple%20Lion%20Release%20WK2%20(Tests)/r123662%20(1421)/results.html

http://build.webkit.org/results/Apple%20Lion%20Release%20WK2%20(Tests)/r123662%20(1421)/compositing/backing/no-backing-for-clip-overlap-crash-log.txt

no crash log found for WebProcess:99181.  Process failed to become responsive before timing out.

Some of the crash reports in the same run just have:

no crash log found for WebProcess:99252.

[Brian Weinstein]

However, in the bot's ~/Library/Logs/DiagnosticReports, I am not seeing any crash reports from cookied, so my comment might be misleading.

[Glenn Adams]

not sure if it is related to this WKB, but I'm noticing failures to read crash_logs on mac-lion with the following errors, which match current output from buildbot [1] (see tail of file):

[1] http://build.webkit.org/builders/Apple%20Lion%20Debug%20WK2%20%28Tests%29/builds/1907/steps/layout-test/logs/stdio

15:25:02.475 73067 looking for crash log for WebProcess:73112

TypeError raised: coercing to Unicode: need string or buffer, tuple found
15:25:02.597 73067     File "/Volumes/Data/slave/lion-debug-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 115, in run
15:25:02.598 73067       unexpected_result_count = manager.run(args)
15:25:02.598 73067     File "/Volumes/Data/slave/lion-debug-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 881, in run
15:25:02.598 73067       self._look_for_new_crash_logs(result_summary, start_time)
15:25:02.598 73067     File "/Volumes/Data/slave/lion-debug-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 964, in _look_for_new_crash_logs
15:25:02.598 73067       writer.write_crash_log(crash_log)
15:25:02.598 73067     File "/Volumes/Data/slave/lion-debug-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer.py", line 167, in write_crash_log
15:25:02.598 73067       self._write_text_file(filename, crash_log)
15:25:02.598 73067     File "/Volumes/Data/slave/lion-debug-tests-wk2/build/Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer.py", line 136, in _write_text_file
15:25:02.598 73067       self._filesystem.write_text_file(path, contents)
15:25:02.598 73067     File "/Volumes/Data/slave/lion-debug-tests-wk2/build/Tools/Scripts/webkitpy/common/system/filesystem.py", line 226, in write_text_file
15:25:02.598 73067       f.write(contents)
15:25:02.598 73067     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 691, in write
15:25:02.599 73067       return self.writer.write(data)
15:25:02.599 73067     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 351, in write
15:25:02.599 73067       data, consumed = self.encode(object, self.errors)
Failed to execute Tools/Scripts/new-run-webkit-tests at ./Tools/Scripts/run-webkit-tests line 124.
program finished with exit code 254

this problem (failure to write crash_log) seems to be fixable by the following patch:

diff --git a/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py b/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
index 0544918..dd19853 100644
--- a/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
+++ b/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
@@ -961,7 +961,7 @@ class Manager(object):
         if crash_logs:
             for test, crash_log in crash_logs.iteritems():
                 writer = TestResultWriter(self._port._filesystem, self._port, self._port.results_directory(), test)
-                writer.write_crash_log(crash_log)
+                writer.write_crash_log(unicode(crash_log))
 
     def _mark_interrupted_tests_as_skipped(self, result_summary):
         for test_name in self._test_files:

[Dirk Pranke]

(In reply to comment #9)
> not sure if it is related to this WKB, but I'm noticing failures to read crash_logs on mac-lion with the following errors, which match current output from buildbot [1] (see tail of file):
> 

Nope, that's an unrelated issue (see bug 92664, which I just fixed in r124071).