Bug 188079 – [ews-build] Add build step to run API Tests

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Aakash Jain]

We should have an EWS for API tests. For that we should have a build step to run API tests.

[Aakash Jain]

Created attachment 345875 [details]
Proposed patch

Sample run: http://ews-build.webkit-uat.org/#/builders/18/builds/26

[Aakash Jain]

Created attachment 345876 [details]
Proposed patch

Sample run: http://ews-build.webkit-uat.org/#/builders/18/builds/26

[EWS Watchlist]

Attachment 345876 [details] did not pass style-queue:


ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:23:  No name 'logobserver' in module 'buildbot.process'  [pylint/E0611] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:123:  [TestWithFailureCount.getResultSummary] Instance of 'TestWithFailureCount' has no 'results' member  [pylint/E1101] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:126:  [TestWithFailureCount.getResultSummary] Instance of 'TestWithFailureCount' has no 'results' member  [pylint/E1101] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:127:  [TestWithFailureCount.getResultSummary] Instance of 'TestWithFailureCount' has no 'results' member  [pylint/E1101] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:892:  [TestRunAPITests.test_success_mac] Passing unexpected keyword argument 'state_string' in function call  [pylint/E1123] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:892:  [TestRunAPITests.test_success_mac] No value passed for parameter 'status_text' in function call  [pylint/E1120] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:918:  [TestRunAPITests.test_success_ios_simulator] Passing unexpected keyword argument 'state_string' in function call  [pylint/E1123] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:918:  [TestRunAPITests.test_success_ios_simulator] No value passed for parameter 'status_text' in function call  [pylint/E1120] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:958:  [TestRunAPITests.test_one_failure] Passing unexpected keyword argument 'state_string' in function call  [pylint/E1123] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:958:  [TestRunAPITests.test_one_failure] No value passed for parameter 'status_text' in function call  [pylint/E1120] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1012:  [TestRunAPITests.test_multiple_failures_and_timeouts] Passing unexpected keyword argument 'state_string' in function call  [pylint/E1123] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1012:  [TestRunAPITests.test_multiple_failures_and_timeouts] No value passed for parameter 'status_text' in function call  [pylint/E1120] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1028:  [TestRunAPITests.test_unexpecte_failure] Passing unexpected keyword argument 'state_string' in function call  [pylint/E1123] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1028:  [TestRunAPITests.test_unexpecte_failure] No value passed for parameter 'status_text' in function call  [pylint/E1120] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1055:  [TestRunAPITests.test_no_failures_or_timeouts_with_disabled] Passing unexpected keyword argument 'state_string' in function call  [pylint/E1123] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1055:  [TestRunAPITests.test_no_failures_or_timeouts_with_disabled] No value passed for parameter 'status_text' in function call  [pylint/E1120] [5]
Total errors found: 16 in 3 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Lucas Forschler]

Comment on attachment 345876 [details]
Proposed patch

View in context: https://bugs.webkit.org/attachment.cgi?id=345876&action=review

r=me as is, with a nit, and something to think about for future patches.

> Tools/BuildSlaveSupport/ews-build/steps.py:378
> +    descriptionDone = ['api-tests']

api tests completed?

> Tools/BuildSlaveSupport/ews-build/steps_unittest.py:871
> +        self.setProperty('fullPlatform', 'mac-sierra')

some pondering...  mac-highsierra is released, and Mojave is in beta. Mac-sierra will be the next platform to fall off the supported list. 
I don't think it really matters for unit tests, unless we run into a problem where the fullPlatform is not in a supported list somewhere...
Do we want to be in the business of updating these periodically as they fall off, or just be ok with the fact that it will soon reference really stale releases?

If we try to keep them relevant, then we should be using mac-mojave everywhere, which would be a larger change in many files.

[Aakash Jain]

Committed r234280: <https://trac.webkit.org/changeset/234280>

[Radar WebKit Bug Importer]

<rdar://problem/42641478>