Bug 44292
| Summary: | Add more instances of Windows EWS | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Jessie Berlin <jberlin> |
| Component: | Tools / Tests | Assignee: | Lucas Forschler <lforschler> |
| Status: | RESOLVED CONFIGURATION CHANGED | ||
| Severity: | Normal | CC: | abarth, eric, jberlin, lforschler |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | PC | ||
| OS: | Windows 7 | ||
| Bug Depends on: | 35460, 45786 | ||
| Bug Blocks: | |||
Jessie Berlin
Often, by the time the Windows EWS has run for a patch, the patch has already been committed. A number of Windows build-breaks could be avoided if it was able to run and finish earlier (often times the build break is caused by something as simple as forgetting to add new files to the .vcproj file when they are being added to the .xcodeproj file).
Right now the Windows-EWS bot is an Amazon EC2 instance owned by Adam Barth to which both Adam Barth and Eric Seidel have admin keys. In the absence of extra hardware (a windows machine on which we don't mind running untrusted code), it would be nice if we could make it so that more than one instance of Windows EWS could be running at the same time.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Adam Barth
Eric already added a lot of the infrastructure for this feature. It's mostly a matter of letting the individual bots "lock" a task on the StatusServer so the other bot takes the next one instead.
Eric Seidel (no email)
The commit-queue just learned how to have more instances as part of bug 45786, moving us one step closer to making this a reality.
Jessie Berlin
(In reply to comment #2)
> The commit-queue just learned how to have more instances as part of bug 45786, moving us one step closer to making this a reality.
What is left?
Eric Seidel (no email)
(In reply to comment #3)
> (In reply to comment #2)
> > The commit-queue just learned how to have more instances as part of bug 45786, moving us one step closer to making this a reality.
>
> What is left?
The EWS systems use a different mechanism for figuring out what patch to run.
All queues use "next_work_item" to get the next patch:
http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/tool/commands/queues.py#L103
However the EWS (through AbstractiReviewQueue) overrides this to use PatchCollection:
http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/tool/commands/queues.py#L399
The commit-queue on the other hand, round trips through the server:
http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/tool/commands/queues.py#L192
(or is about to after the patch on bug 45786).
We need to teach the EWS how to round trip through the server, and may need to improve the server side code to avoid race conditions between clients. RIght now there are race conditions in the logic I added with bug 45786, but for the commit-queue it doesn't really matter.
Eventually the plan is to separate out the bugzilla polling from the EWS queue into some sort of "feeder" queue, whereby which all of the EWS bots just read their next patch from the status server (instead of reading all the patches from bugzilla and then informing the status server). That would greatly reduce the load on bugzilla. We haven't done it yet because at the current 2 minute poll time for the EWS bots the bugzilla load is tiny (at least according to wms). Eventually we'd like to have a single queue whose job it is to poll bugzilla and tell all the other queues what they're next to do. But that may be a yak we shave after implementing some simple server-side mediation between multiple bots, just like I did in bug 45786 for the commit-queue.
I'll look at solving this tomorrow.
Eric Seidel (no email)
The architecture is now ready after bug 35460. It's now just a question of hardware.
Jessie Berlin
(In reply to comment #5)
> The architecture is now ready after bug 35460. It's now just a question of hardware.
What hardware is needed?
Eric Seidel (no email)
Adding a new EWS bot for any port is as simple as checking out webkit and running:
webkit-patch win-ews (or whatever EWS you want to run)
So any machine will do. We even have some wrapper scripts in WebKitTools/EWSTools to make starting bots easy.
Adam and I might buy an extra windows EC2 instance to run another win-ews bot, we'll see.
First, I need to sort out the current EWS-bot sickness caused by my r+ fixes (see the webkit-dev mail on the subject). I'll update this bug when I believe starting an EWS will be back to being a no-trouble affair (should be monday).
Lucas Forschler
Apple now has 7 Windows EWS bots up and running.
We'd like to enable running tests on windows EWS. I am searching to see if there is another bug tracking that. I think we can close this one out.