Bug 111792 - EWS keeps trying the same patch
Summary: EWS keeps trying the same patch
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-07 16:35 PST by Ryosuke Niwa
Modified: 2013-03-10 08:40 PDT (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ryosuke Niwa 2013-03-07 16:35:41 PST
e.g.
http://webkit-commit-queue.appspot.com/queue-status/mac-ews/bots/webkit-ews-03
Patch 191803 from bug 111459:
1 hour, 10 minutes ago (webkit-ews-03) Pass
1 hour, 10 minutes ago (webkit-ews-03) Unable to pass tests without patch (tree is red?) [results] http://webkit-commit-queue.appspot.com/results/17024050
1 hour, 23 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17074126
1 hour, 32 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17059143
2 hours, 21 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17038094
2 hours, 45 minutes ago (webkit-ews-03) Patch does not pass tests [results] http://webkit-commit-queue.appspot.com/results/17015236


http://webkit-commit-queue.appspot.com/results/17024050

Regressions: Unexpected text-only failures (3)
  fast/borders/border-fit-2.html [ Failure ]
  fast/borders/border-fit.html [ Failure ]
  fast/repaint/border-fit-lines.html [ Failure ]

Regressions: Unexpected crashes (1)
  editing/selection/selection-modify-crash.html [ Crash ]


http://webkit-commit-queue.appspot.com/results/17074126
Regressions: Unexpected text-only failures (3)
  fast/borders/border-fit-2.html [ Failure ]
  fast/borders/border-fit.html [ Failure ]
  fast/repaint/border-fit-lines.html [ Failure ]

http://webkit-commit-queue.appspot.com/results/17059143
Regressions: Unexpected text-only failures (3)
  fast/borders/border-fit-2.html [ Failure ]
  fast/borders/border-fit.html [ Failure ]
  fast/repaint/border-fit-lines.html [ Failure ]

Why doesn't EWS stop at 17059143? Why is it trying two more runs?
Comment 1 Alexey Proskuryakov 2013-03-09 21:09:42 PST
Another example: <http://queues.webkit.org/patch/192273> had 46 test runs already.

On another patch, I saw that EWS tried 20 times and then gave up, saying that tests don't pass without the patch, and showing a green bubble (!!!).

When does EWS decide to give up?
Comment 2 Adam Barth 2013-03-10 08:38:44 PDT
> On another patch, I saw that EWS tried 20 times and then gave up, saying that tests don't pass without the patch, and showing a green bubble (!!!).

That's normal behavior.  I would bet that the patch doesn't cause any new tests to fail.

> When does EWS decide to give up?

When it determines that a patch either does or does not cause any new failures.  It can also give up if it encounters some other sorts of errors, such as the patch not applying to top-of-tree.
Comment 3 Adam Barth 2013-03-10 08:40:49 PDT
If you're seeing a lot of retries, there are likely two causes:

1) Flaky tests.  If the bot doesn't get repeatable test results (e.g., when running the same set of tests on the same binary), then it retries.

2) Red tree.  If there are too many test failures in the tree, the bot will wait until fewer tests are failing so that it's getting meaningful data.  The limit is hardcoded and something like 20 or 50 failing tests.

I suspect that your problems are caused by flaky tests.