Bug 141235 – Tests don't work on some bots: Failed to stop wptwk

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Alexey Proskuryakov]

I see tests fail on multiple bots tonight:

https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK2%20(Tests)/builds/11276
https://build.webkit.org/builders/Apple%20Mavericks%20Debug%20WK2%20(Tests)/builds/9271

ServerError: Failed to stop wptwk: please go to web platform test server log file to get the PID list

[youenn fablet]

OK.
I will mark as failed the two tests so that it does not disturb the bots and will investigate this further.

[Alexey Proskuryakov]

The bots are still completely broken - it's not just the two tests, it's an exception in run-webkit-tests that blocks reporting any results at all.

[youenn fablet]

Created attachment 246043 [details]
Patch

[youenn fablet]

(In reply to comment #3)
> Created attachment 246043 [details]
> Patch

This patch fixes the issue that broke the bots (uncaught exception).

The underlying issue is that run-webkit-tests is not always able to stop the wpt server properly. Reasons are unclear to me.

To make things more robust, this patch kills not only the launcher process but also all server processes. This should allow clean restarting.

[youenn fablet]

(In reply to comment #4)
> (In reply to comment #3)
> > Created attachment 246043 [details]
> > Patch
> 
> The underlying issue is that run-webkit-tests is not always able to stop the
> wpt server properly. Reasons are unclear to me.

I looked at the logs.
The bots stopped in the middle of layout test runs.
My previous patch was not handling the recovery correctly :(
The uploaded fix for bug 141235 should handle that issue, and probably a bit more.
 
It may be interesting to do "ps -A | grep python" on one or both of the corrupted bots when they do not run any task.
There may be 4 or 5 python processes remaining, with ids close one to the others. It might be best to kill them or restart the bots.

The current patch should be able to cleanly kill such processes in the case of a layout test run failure.

[youenn fablet]

Created attachment 246640 [details]
Adding unit tests

[Ryosuke Niwa]

Comment on attachment 246640 [details]
Adding unit tests

View in context: https://bugs.webkit.org/attachment.cgi?id=246640&action=review

Looks sane to me.

> Tools/ChangeLog:9
> +        Added killing of server subprocesses if still running.

Just say "kill the server wpt subprocess " in stop().

> Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_server.py:67
> +        self._servers_file = self._filesystem.join(self._runtime_path, '%s_servers.json' % (self._name))

Can't we just create a temporary file instead?

[youenn fablet]

Thanks for the review.

> > Tools/ChangeLog:9
> > +        Added killing of server subprocesses if still running.
> 
> Just say "kill the server wpt subprocess " in stop().

OK

> > Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_server.py:67
> > +        self._servers_file = self._filesystem.join(self._runtime_path, '%s_servers.json' % (self._name))
> 
> Can't we just create a temporary file instead?

self._runtime_path is a temp dir in which one can find the pid file for apache server or wpt server launcher.

[youenn fablet]

Created attachment 247051 [details]
Updated changelog

[WebKit Commit Bot]

Comment on attachment 247051 [details]
Updated changelog

Clearing flags on attachment: 247051

Committed r180480: <http://trac.webkit.org/changeset/180480>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.