RESOLVED FIXED 204707
Add a unit-test to EWS for scenario when there is flakiness with patch and also on clean tree
https://bugs.webkit.org/show_bug.cgi?id=204707
Summary Add a unit-test to EWS for scenario when there is flakiness with patch and al...
Aakash Jain
Reported 2019-11-29 10:06:01 PST
Add a unit-test to old EWS for scenario when there is flakiness with patch and also on clean tree.
Attachments
Unit-test for old EWS (1.79 KB, patch)
2019-11-29 10:08 PST, Aakash Jain
no flags
Similar unit-test for new EWS (1.98 KB, patch)
2019-11-29 10:16 PST, Aakash Jain
no flags
Aakash Jain
Comment 1 2019-11-29 10:08:16 PST
Created attachment 384518 [details] Unit-test for old EWS
Aakash Jain
Comment 2 2019-11-29 10:09:51 PST
Radar WebKit Bug Importer
Comment 3 2019-11-29 10:10:18 PST
Aakash Jain
Comment 4 2019-11-29 10:16:59 PST
Created attachment 384519 [details] Similar unit-test for new EWS
Aakash Jain
Comment 5 2019-11-29 10:18:13 PST
.
Aakash Jain
Comment 6 2019-11-29 10:20:48 PST
Alexey Proskuryakov
Comment 7 2019-11-29 22:06:53 PST
I think that I know why I didn't believe that this was the old EWS behavior. Looking at the "if self._results_failed_different_tests" branch in PatchAnalysisTask._retry_layout_tests, I see that it calls self._should_defer_patch_or_throw, which says that the result should be a PASS. But then we ignore the result, and make it a DEFER anyway! if tests_that_consistently_failed: if self._should_defer_patch_or_throw(tests_that_consistently_failed, first_results_archive, first_script_error, first_failure_status_id): return False # Defer patch # At this point we know that at least one test flaked, but no consistent failures # were introduced. This is a bit of a grey-zone. return False # Defer patch What's the point of the first "return False" here if we are going to defer either way?
Note You need to log in before you can comment on or make changes to this bug.