In our EWS configuration, we share bots over multiple queues to improve the efficiency. Sometimes a bot might be connected over builder and tester queues. If there is pending build-requests over both the queues, we want to prioritize builder queue over tester. "By default, buildbot will attempt to start builds on builders in order, beginning with the builder with the oldest pending request." https://docs.buildbot.net/current/manual/configuration/global.html#prioritizing-builders For e.g.: 'macOS-High-Sierra-Debug-Build-EWS' and 'macOS-High-Sierra-Debug-WK1-Tests-EWS' share bots ews113 - ews117. Quite often it happens that there are pending build-requests on both the queues (especially when there are test failures on trunk and tester queue is backlogged). By default Buildbot prioritize the queue having older build-request. Because of that, most of the bots keep processing tester queue, and builder queue takes low priority. However we want builder queues to be prioritized over tester queues. This will also enable to us to do more bot sharing between builder and tester queues, and have higher utilization of our bots.
Created attachment 383135 [details] Patch
Reference: https://docs.buildbot.net/current/manual/customization.html#builder-priority-functions Tested on UAT instance using webkit-misc bot.
Comment on attachment 383135 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=383135&action=review > Tools/BuildSlaveSupport/ews-build/loadConfig.py:80 > + c['prioritizeBuilders'] = prioritizeBuilders Won't this mean that any build request will pre-empt a test request? Is that actually what we want?
(In reply to Jonathan Bedard from comment #3) > Won't this mean that any build request will pre-empt a test request? Yes, but only for bots which are shared on multiple queues. Currently each queue has several dedicated bots. Some of the bots are shared between builder and tester queue. Currently reverse is happening with the shared bots, especially on 'macOS-High-Sierra-Debug-WK1-Tests-EWS' which shares a lot of bots with 'macOS-High-Sierra-Debug-Build-EWS'. > Is that actually what we want? Yes, I believe that is desirable. Also note that builders are usually much faster than testers, since builders do incremental builds.
(In reply to Aakash Jain from comment #4) > (In reply to Jonathan Bedard from comment #3) > > Won't this mean that any build request will pre-empt a test request? > Yes, but only for bots which are shared on multiple queues. Currently each > queue has several dedicated bots. Some of the bots are shared between > builder and tester queue. > > Currently reverse is happening with the shared bots, especially on > 'macOS-High-Sierra-Debug-WK1-Tests-EWS' which shares a lot of bots with > 'macOS-High-Sierra-Debug-Build-EWS'. > > > Is that actually what we want? > Yes, I believe that is desirable. Also note that builders are usually much > faster than testers, since builders do incremental builds. Seems reasonable, but we should keep an eye on this, it seems like a potential source of issues
Committed r252244: <https://trac.webkit.org/changeset/252244>
<rdar://problem/57027388>