Add support for webkitpy tests EWS

[Aakash Jain]

We should have a EWS which runs webkitpy tests. It would help catch webkitpy failures early-on.

See <rdar://problem/18231414>.

[Aakash Jain]

Created attachment 324268 [details]
Proposed patch

[Aakash Jain]

Tested on an staging instance.

[Aakash Jain]

This patch is similar to bindings-test EWS patches, combination of https://bugs.webkit.org/show_bug.cgi?id=168979, https://bugs.webkit.org/show_bug.cgi?id=169063, https://bugs.webkit.org/show_bug.cgi?id=169308

[Daniel Bates]

Comment on attachment 324268 [details]
Proposed patch

View in context: https://bugs.webkit.org/attachment.cgi?id=324268&action=review

> Tools/Scripts/webkitpy/common/config/ews.json:62
> +    "Webkitpy EWS": {

Webkitpy => webkitpy OR WebKitPy

> Tools/Scripts/webkitpy/common/net/webkitpytestresults.py:30
> +class BindingsTestResults(AbstractTestResults):

Please name the file after this class or vice versa.

> Tools/Scripts/webkitpy/common/net/webkitpytestresults_unittest.py:28
> +class BindingsTestResultsTest(unittest.TestCase):

Ditto.

> Tools/Scripts/webkitpy/tool/bot/webkitpytestresultsreader.py:31
> +class BindingsTestResultsReader(AbstractTestResultsReader):

Ditto.

> Tools/Scripts/webkitpy/tool/bot/webkitpytestresultsreader.py:31
> +class WebkitpyTestResultsReader(AbstractTestResultsReader):

Ditto.

> Tools/Scripts/webkitpy/tool/commands/earlywarningsystem.py:67
> +            self._test_results_reader = WebkitpyTestResultsReader(self._tool, self._port.jsc_results_directory())

We should rename jsc_results_directory() to something more generic since we use this directory for other test results.

> Tools/Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py:204
> +            'webkitpy-ews',

Please add this to the list in sorted order according to the UNIX sort command.

> Tools/Scripts/webkitpy/tool/steps/runtests.py:84
> +        # FIXME: Remove this code since we now have a separate webkitpy EWS.

We talked about this in person. It does not seem harmful to support running these tests when using webkit-patch interactively.

> Tools/Scripts/webkitpy/tool/steps/runtests.py:187
> +        args.append("--json-output=%s" % results_file_path)

I believe we prefer String.format() over % operator.

[Aakash Jain]

Created attachment 325789 [details]
Updated patch

Review comments incorporated. Removed some code duplication.

Also enabled checking for patch relevance so that webkitpy-ews bubble only shows for relevant patches.

Tested on staging instance.

[WebKit Commit Bot]

Comment on attachment 325789 [details]
Updated patch

Clearing flags on attachment: 325789

Committed r224547: <https://trac.webkit.org/changeset/224547>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Aakash Jain]

Seems to be working fine: https://webkit-queues.webkit.org/queue-status/webkitpy-ews