Summary: | [ews] Add build step to Analyze JSC Tests Results | ||||||
---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Aakash Jain <aakash_jain> | ||||
Component: | Tools / Tests | Assignee: | Aakash Jain <aakash_jain> | ||||
Status: | RESOLVED FIXED | ||||||
Severity: | Normal | CC: | aakash_jain, ap, commit-queue, jbedard, pmatos, webkit-bug-importer, zhifei_fang | ||||
Priority: | P2 | Keywords: | InRadar | ||||
Version: | Other | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
See Also: | https://bugs.webkit.org/show_bug.cgi?id=204179 | ||||||
Attachments: |
|
Description
Aakash Jain
2019-11-13 15:33:08 PST
Created attachment 383502 [details]
Patch
Sample runs: JSC stress test failure: https://ews-build.webkit-uat.org/#/builders/21/builds/430 binary and stress test failure: https://ews-build.webkit-uat.org/#/builders/21/builds/442 Flaky tests: https://ews-build.webkit-uat.org/#/builders/21/builds/435 Pre-existing failure: https://ews-build.webkit-uat.org/#/builders/21/builds/457 Comment on attachment 383502 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=383502&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:1081 > + flaky_binary_failures = first_run_binary_failures.union(second_run_binary_failures) - first_run_binary_failures.intersection(second_run_binary_failures) I'm not convinced that we should give a green bubble with flaky binary failures....From talking to Zhifei, I get the sense that the binaries don't have the same sort of flake problems that layout tests or jsc stress tests have. wait, if we have already submitted results to results database, why we need to rerun it without patch ? (In reply to Zhifei Fang from comment #4) > wait, if we have already submitted results to results database, why we need > to rerun it without patch ? We don't have EWS plugged in to the results database yet. We totally could now, though. Ling recently sorted out the network access. Comment on attachment 383502 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=383502&action=review >> Tools/BuildSlaveSupport/ews-build/steps.py:1081 >> + flaky_binary_failures = first_run_binary_failures.union(second_run_binary_failures) - first_run_binary_failures.intersection(second_run_binary_failures) > > I'm not convinced that we should give a green bubble with flaky binary failures....From talking to Zhifei, I get the sense that the binaries don't have the same sort of flake problems that layout tests or jsc stress tests have. Aakash, Zhifei and I talked about this a bit offline. Binary tests probably aren't flakey, but this is what old EWS did and we don't actually have concrete data on if test binaries are flakey without https://bugs.webkit.org/show_bug.cgi?id=204091. We might come back and revisit this, but for the sake of converting JSC EWS to new EWS, this behavior is the same as what we were doing before. Comment on attachment 383502 [details] Patch Clearing flags on attachment: 383502 Committed r252446: <https://trac.webkit.org/changeset/252446> All reviewed patches have been landed. Closing bug. |