RESOLVED FIXED 188156
[ews-build] Add support for API tests in OpenSource EWS 
https://bugs.webkit.org/show_bug.cgi?id=188156
Summary [ews-build] Add support for API tests in OpenSource EWS
Aakash Jain
Reported 2018-07-29 23:33:39 PDT
We should add support for running API tests in the new OpenSource EWS Buildbot, both for mac as well as ios-simulator. Note that these tests require building. So, a builder should trigger these tests appropriately. Also see: <rdar://problem/13690874> and <rdar://problem/30214936>.
Attachments
Proposed patch (7.62 KB, patch)
2018-07-29 23:43 PDT, Aakash Jain 		lforschler: review+
DetailsFormatted DiffDiff
Archive of layout-test-results from ews206 for win-future (13.01 MB, application/zip)
2018-07-30 01:58 PDT, EWS Watchlist 		no flags
Details
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
Aakash Jain
Comment 1 2018-07-29 23:43:03 PDT
Created attachment 346051 [details] Proposed patch Sample run: http://ews-build.webkit-uat.org/#/builders/20/builds/2
EWS Watchlist
Comment 2 2018-07-30 01:57:59 PDT Comment hidden (obsolete)
EWS Watchlist
Comment 3 2018-07-30 01:58:11 PDT Comment hidden (obsolete)
Aakash Jain
Comment 4 2018-07-30 12:10:06 PDT
Sample run for macOS API Tests: http://ews-build.webkit-uat.org/#/builders/19/builds/5
Aakash Jain
Comment 5 2018-07-30 16:30:40 PDT
Committed r234397: <https://trac.webkit.org/changeset/234397>
Radar WebKit Bug Importer
Comment 6 2018-07-30 16:32:26 PDT
<rdar://problem/42751658>
Aakash Jain
Comment 7 2018-07-31 17:11:30 PDT
Note: Since the API tests are flaky. I would also need to add logic to: - retry these tests on failure - modify run_api_tests script to output test results in json format - parse and compare results and decide whether the failures are introduced by the patch
Jonathan Bedard
Comment 8 2018-08-22 14:31:34 PDT
(In reply to Aakash Jain from comment #7) > Note: Since the API tests are flaky. I would also need to add logic to: > - retry these tests on failure > - modify run_api_tests script to output test results in json format > - parse and compare results and decide whether the failures are introduced > by the patch Wait, which API tests are flaky? The rest of our infrastructure generally assumes they aren't (this isn't totally true, but it's mostly true)
Note You need to log in before you can comment on or make changes to this bug.