We should have an EWS for API tests. For that we should have a build step to run API tests.
Created attachment 345875 [details] Proposed patch Sample run: http://ews-build.webkit-uat.org/#/builders/18/builds/26
Created attachment 345876 [details] Proposed patch Sample run: http://ews-build.webkit-uat.org/#/builders/18/builds/26
Attachment 345876 [details] did not pass style-queue: ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:23: No name 'logobserver' in module 'buildbot.process' [pylint/E0611] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:123: [TestWithFailureCount.getResultSummary] Instance of 'TestWithFailureCount' has no 'results' member [pylint/E1101] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:126: [TestWithFailureCount.getResultSummary] Instance of 'TestWithFailureCount' has no 'results' member [pylint/E1101] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps.py:127: [TestWithFailureCount.getResultSummary] Instance of 'TestWithFailureCount' has no 'results' member [pylint/E1101] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:892: [TestRunAPITests.test_success_mac] Passing unexpected keyword argument 'state_string' in function call [pylint/E1123] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:892: [TestRunAPITests.test_success_mac] No value passed for parameter 'status_text' in function call [pylint/E1120] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:918: [TestRunAPITests.test_success_ios_simulator] Passing unexpected keyword argument 'state_string' in function call [pylint/E1123] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:918: [TestRunAPITests.test_success_ios_simulator] No value passed for parameter 'status_text' in function call [pylint/E1120] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:958: [TestRunAPITests.test_one_failure] Passing unexpected keyword argument 'state_string' in function call [pylint/E1123] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:958: [TestRunAPITests.test_one_failure] No value passed for parameter 'status_text' in function call [pylint/E1120] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1012: [TestRunAPITests.test_multiple_failures_and_timeouts] Passing unexpected keyword argument 'state_string' in function call [pylint/E1123] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1012: [TestRunAPITests.test_multiple_failures_and_timeouts] No value passed for parameter 'status_text' in function call [pylint/E1120] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1028: [TestRunAPITests.test_unexpecte_failure] Passing unexpected keyword argument 'state_string' in function call [pylint/E1123] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1028: [TestRunAPITests.test_unexpecte_failure] No value passed for parameter 'status_text' in function call [pylint/E1120] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1055: [TestRunAPITests.test_no_failures_or_timeouts_with_disabled] Passing unexpected keyword argument 'state_string' in function call [pylint/E1123] [5] ERROR: Tools/BuildSlaveSupport/ews-build/steps_unittest.py:1055: [TestRunAPITests.test_no_failures_or_timeouts_with_disabled] No value passed for parameter 'status_text' in function call [pylint/E1120] [5] Total errors found: 16 in 3 files If any of these errors are false positives, please file a bug against check-webkit-style.
Comment on attachment 345876 [details] Proposed patch View in context: https://bugs.webkit.org/attachment.cgi?id=345876&action=review r=me as is, with a nit, and something to think about for future patches. > Tools/BuildSlaveSupport/ews-build/steps.py:378 > + descriptionDone = ['api-tests'] api tests completed? > Tools/BuildSlaveSupport/ews-build/steps_unittest.py:871 > + self.setProperty('fullPlatform', 'mac-sierra') some pondering... mac-highsierra is released, and Mojave is in beta. Mac-sierra will be the next platform to fall off the supported list. I don't think it really matters for unit tests, unless we run into a problem where the fullPlatform is not in a supported list somewhere... Do we want to be in the business of updating these periodically as they fall off, or just be ok with the fact that it will soon reference really stale releases? If we try to keep them relevant, then we should be using mac-mojave everywhere, which would be a larger change in many files.
Committed r234280: <https://trac.webkit.org/changeset/234280>
<rdar://problem/42641478>