Bug 48943 – run-webkit-tests thinks the web process crashed if WebKitTestRunner fails to reset it to a consistent state

• Status: NEW
• Component: Tools / Tests

[Adam Roben (:aroben)]

In the fix for bug 43047 I made WebKitTestRunner exit if it fails to reset the web process to a consistent state between tests. When WebKitTestRunner exits, run-webkit-tests assumes it must have crashed, and counts whatever the current test is as a crash. This is confusing!

[Adam Roben (:aroben)]

Perhaps we should count the failure to reset the web process against whatever the last-run test was, since presumably it is the one that screwed things up so the web process couldn't be reset.

[Sam Weinig]

(In reply to comment #1)
> Perhaps we should count the failure to reset the web process against whatever the last-run test was, since presumably it is the one that screwed things up so the web process couldn't be reset.

Yeah, that sounds like the correct solution here, then we want the script to re-run the current test.

[Adam Roben (:aroben)]

(In reply to comment #1)
> Perhaps we should count the failure to reset the web process against whatever the last-run test was, since presumably it is the one that screwed things up so the web process couldn't be reset.

One way to accomplish this would be to reset the web process at the end of each test instead of at the beginning, and to wait to dump the test results until the reset is completed. Then, if the reset fails, we could dump an extra message into the test results to indicate the failure, perhaps followed by dumping some special string that would tell run-webkit-tests to relaunch WTR.

[Adam Roben (:aroben)]

<rdar://problem/8630445>

[Adam Roben (:aroben)]

Maciej just changed this behavior in r76178 (see bug 52757). Maciej, can you please update the status here? WHat do you think about the idea Sam and I discussed above?

[Adam Roben (:aroben)]

r76178 just changed things so that run-webkit-tests now thinks the web process crashed.