Bug 111792 – EWS keeps trying the same patch

• Status: NEW
• Component: Tools / Tests

[Ryosuke Niwa]

e.g.
http://webkit-commit-queue.appspot.com/queue-status/mac-ews/bots/webkit-ews-03
Patch 191803 from bug 111459:
1 hour, 10 minutes ago (webkit-ews-03) Pass
1 hour, 10 minutes ago (webkit-ews-03) Unable to pass tests without patch (tree is red?) [results] http://webkit-commit-queue.appspot.com/results/17024050
1 hour, 23 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17074126
1 hour, 32 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17059143
2 hours, 21 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17038094
2 hours, 45 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17015236


http://webkit-commit-queue.appspot.com/results/17024050

Regressions: Unexpected text-only failures (3)
  fast/borders/border-fit-2.html [ Failure ]
  fast/borders/border-fit.html [ Failure ]
  fast/repaint/border-fit-lines.html [ Failure ]

Regressions: Unexpected crashes (1)
  editing/selection/selection-modify-crash.html [ Crash ]


http://webkit-commit-queue.appspot.com/results/17074126
Regressions: Unexpected text-only failures (3)
  fast/borders/border-fit-2.html [ Failure ]
  fast/borders/border-fit.html [ Failure ]
  fast/repaint/border-fit-lines.html [ Failure ]

http://webkit-commit-queue.appspot.com/results/17059143
Regressions: Unexpected text-only failures (3)
  fast/borders/border-fit-2.html [ Failure ]
  fast/borders/border-fit.html [ Failure ]
  fast/repaint/border-fit-lines.html [ Failure ]

Why doesn't EWS stop at 17059143? Why is it trying two more runs?

[Alexey Proskuryakov]

Another example: <http://queues.webkit.org/patch/192273> had 46 test runs already.

On another patch, I saw that EWS tried 20 times and then gave up, saying that tests don't pass without the patch, and showing a green bubble (!!!).

When does EWS decide to give up?

[Adam Barth]

> On another patch, I saw that EWS tried 20 times and then gave up, saying that tests don't pass without the patch, and showing a green bubble (!!!).

That's normal behavior.  I would bet that the patch doesn't cause any new tests to fail.

> When does EWS decide to give up?

When it determines that a patch either does or does not cause any new failures.  It can also give up if it encounters some other sorts of errors, such as the patch not applying to top-of-tree.

[Adam Barth]

If you're seeing a lot of retries, there are likely two causes:

1) Flaky tests.  If the bot doesn't get repeatable test results (e.g., when running the same set of tests on the same binary), then it retries.

2) Red tree.  If there are too many test failures in the tree, the bot will wait until fewer tests are failing so that it's getting meaningful data.  The limit is hardcoded and something like 20 or 50 failing tests.

I suspect that your problems are caused by flaky tests.