Bug 188516 – [ews-build] Separate queues for Builders and Testers - macOS Sierra

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Aakash Jain]

Currently EWS bots build and test on same machine. This is not very efficient. We should instead build (and upload) the archive and re-use it for multiple types of tests (e.g.: WK1, WK2, API tests). This would improve the overall system efficiency by reducing the number of builds while producing same kind of output. This would also improve the overall turn-around time to get EWS results after an engineer uploads a patch.

This bug tracks splitting macOS Sierra Release queues in builders and testers.

[Aakash Jain]

Created attachment 347013 [details]
Proposed patch

This change results in a clear efficiency improvement. Instead of Building (compiling WebKit) on multiple queues (WK1 and WK2), Build would be done only on one queue (Builder) and it would be re-used on 3 queues (WK1, WK2, API-tests).

For each patch, we are saving WebKit compilation time twice. This would results in bots being more available and reduce the overall time engineers have to wait to get the EWS results after a patch is uploaded.

Sample runs:

macOS-Sierra-Release-Build-EWS: http://ews-build.webkit-uat.org/#/builders/25/builds/11
macOS-Sierra-Release-WK1-Tests-EWS: http://ews-build.webkit-uat.org/#/builders/26/builds/13
macOS-Sierra-Release-WK2-Tests-EWS: http://ews-build.webkit-uat.org/#/builders/27/builds/12

[EWS Watchlist]

Attachment 347013 [details] did not pass style-queue:


ERROR: Tools/BuildSlaveSupport/ews-build/factories.py:139:  [MacWK1Factory] Undefined variable 'RunWebKit1Tests'  [pylint/E0602] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/factories.py:143:  [MacWK2Factory] Undefined variable 'RunWebKitTests'  [pylint/E0602] [5]
Total errors found: 2 in 3 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Lucas Forschler]

Comment on attachment 347013 [details]
Proposed patch

Mac OS is now called macOS...I think we should probably use 'macOS' everywhere, rather than 'Mac'. I could be convinced to use MacOS (capitol M) since it is the first letter of the class name. But, I would not oppose using lowercase. others may have a stronger opinion. Please ask around and see what others think.

Also, let's either fix the style errors, or override them for buildbot instances if they are false.

[Aakash Jain]

(In reply to Lucas Forschler from comment #3)
> Mac OS is now called macOS...I think we should probably use 'macOS' everywhere, rather than 'Mac'.
Good point. I have changed to macOS everywhere in updated patch.

> Also, let's either fix the style errors, or override them for buildbot instances if they are false.
Filed https://bugs.webkit.org/show_bug.cgi?id=188584 to track style checker issue.

[Aakash Jain]

Created attachment 347119 [details]
Updated patch

[EWS Watchlist]

Attachment 347119 [details] did not pass style-queue:


ERROR: Tools/BuildSlaveSupport/ews-build/factories.py:139:  [macOSWK1Factory] Undefined variable 'RunWebKit1Tests'  [pylint/E0602] [5]
ERROR: Tools/BuildSlaveSupport/ews-build/factories.py:143:  [macOSWK2Factory] Undefined variable 'RunWebKitTests'  [pylint/E0602] [5]
Total errors found: 2 in 3 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Lucas Forschler]

(In reply to Aakash Jain from comment #4)
> (In reply to Lucas Forschler from comment #3)
> > Mac OS is now called macOS...I think we should probably use 'macOS' everywhere, rather than 'Mac'.
> Good point. I have changed to macOS everywhere in updated patch.

can you file a bug to update other instances outside of this patch?

> 
> > Also, let's either fix the style errors, or override them for buildbot instances if they are false.
> Filed https://bugs.webkit.org/show_bug.cgi?id=188584 to track style checker
> issue.

[Aakash Jain]

> can you file a bug to update other instances outside of this patch?
Done in: https://bugs.webkit.org/show_bug.cgi?id=188540

[WebKit Commit Bot]

Comment on attachment 347119 [details]
Updated patch

Clearing flags on attachment: 347119

Committed r234951: <https://trac.webkit.org/changeset/234951>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/43394566>