Bug 196947

Summary: ews-build should clearly indicate flaky test failures
Product: WebKit Reporter: Aakash Jain <aakash_jain>
Component: Tools / TestsAssignee: Aakash Jain <aakash_jain>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, ap, dewei_zhu, lforschler, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch lforschler: review+

Description Aakash Jain 2019-04-15 16:08:41 PDT 
API tests has various flaky failures. Since ews-build runs API tests multiple times, it can notice flaky tests. ews-build should clearly indicate flaky test in the UI when it identifies a flaky test. This would help in easily identifying flaky tests and getting them fixed.
Comment 1 Aakash Jain 2019-04-15 16:09:49 PDT 
Created attachment 367473 [details]
Patch
Comment 2 Lucas Forschler 2019-04-15 16:33:15 PDT 
Comment on attachment 367473 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=367473&action=review

r=me after checking the first/second variable name.

> Tools/BuildSlaveSupport/ews-build/steps.py:802
> +        self._addToLog('stderr', '\nFailures in API Test second run: {}'.format(first_run_failures))

I think this should be second_run_failures.
Comment 3 Aakash Jain 2019-04-15 16:36:28 PDT 
Committed r244306: <https://trac.webkit.org/changeset/244306>
Comment 4 Radar WebKit Bug Importer 2019-04-15 16:37:18 PDT 
<rdar://problem/49923241>