204007 2019-11-08 09:49:21 -0800 [ews] Prioritize builders over testers when a bot is shared over multiple queues 2019-11-08 10:54:20 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap jbedard webkit-bug-importer oldest_to_newest 1588684 0 aakash_jain 2019-11-08 09:49:21 -0800 In our EWS configuration, we share bots over multiple queues to improve the efficiency. Sometimes a bot might be connected over builder and tester queues. If there is pending build-requests over both the queues, we want to prioritize builder queue over tester. "By default, buildbot will attempt to start builds on builders in order, beginning with the builder with the oldest pending request." https://docs.buildbot.net/current/manual/configuration/global.html#prioritizing-builders For e.g.: 'macOS-High-Sierra-Debug-Build-EWS' and 'macOS-High-Sierra-Debug-WK1-Tests-EWS' share bots ews113 - ews117. Quite often it happens that there are pending build-requests on both the queues (especially when there are test failures on trunk and tester queue is backlogged). By default Buildbot prioritize the queue having older build-request. Because of that, most of the bots keep processing tester queue, and builder queue takes low priority. However we want builder queues to be prioritized over tester queues. This will also enable to us to do more bot sharing between builder and tester queues, and have higher utilization of our bots. 1588686 1 383135 aakash_jain 2019-11-08 09:51:42 -0800 Created attachment 383135 Patch 1588689 2 aakash_jain 2019-11-08 09:52:45 -0800 Reference: https://docs.buildbot.net/current/manual/customization.html#builder-priority-functions Tested on UAT instance using webkit-misc bot. 1588710 3 383135 jbedard 2019-11-08 10:23:28 -0800 Comment on attachment 383135 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=383135&action=review > Tools/BuildSlaveSupport/ews-build/loadConfig.py:80 > + c['prioritizeBuilders'] = prioritizeBuilders Won't this mean that any build request will pre-empt a test request? Is that actually what we want? 1588716 4 aakash_jain 2019-11-08 10:38:25 -0800 (In reply to Jonathan Bedard from comment #3) > Won't this mean that any build request will pre-empt a test request? Yes, but only for bots which are shared on multiple queues. Currently each queue has several dedicated bots. Some of the bots are shared between builder and tester queue. Currently reverse is happening with the shared bots, especially on 'macOS-High-Sierra-Debug-WK1-Tests-EWS' which shares a lot of bots with 'macOS-High-Sierra-Debug-Build-EWS'. > Is that actually what we want? Yes, I believe that is desirable. Also note that builders are usually much faster than testers, since builders do incremental builds. 1588717 5 jbedard 2019-11-08 10:49:06 -0800 (In reply to Aakash Jain from comment #4) > (In reply to Jonathan Bedard from comment #3) > > Won't this mean that any build request will pre-empt a test request? > Yes, but only for bots which are shared on multiple queues. Currently each > queue has several dedicated bots. Some of the bots are shared between > builder and tester queue. > > Currently reverse is happening with the shared bots, especially on > 'macOS-High-Sierra-Debug-WK1-Tests-EWS' which shares a lot of bots with > 'macOS-High-Sierra-Debug-Build-EWS'. > > > Is that actually what we want? > Yes, I believe that is desirable. Also note that builders are usually much > faster than testers, since builders do incremental builds. Seems reasonable, but we should keep an eye on this, it seems like a potential source of issues 1588718 6 aakash_jain 2019-11-08 10:53:51 -0800 Committed r252244: <https://trac.webkit.org/changeset/252244> 1588721 7 webkit-bug-importer 2019-11-08 10:54:20 -0800 <rdar://problem/57027388> 383135 2019-11-08 09:51:42 -0800 2019-11-08 10:48:11 -0800 Patch bug-204007-20191108125141.patch text/plain 1682 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1MjIzOSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDE5LTExLTA4ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBQcmlvcml0aXplIGJ1aWxkZXJzIG92ZXIgdGVzdGVycyB3aGVu IGEgYm90IGlzIHNoYXJlZCBvdmVyIG11bHRpcGxlIHF1ZXVlcworICAgICAgICBodHRwczovL2J1 Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjA0MDA3CisKKyAgICAgICAgUmV2aWV3ZWQg YnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVp bGQvbG9hZENvbmZpZy5weToKKyAgICAgICAgKGxvYWRCdWlsZGVyQ29uZmlnKToKKyAgICAgICAg KHByaW9yaXRpemVCdWlsZGVycyk6IE1ldGhvZCB0byBwcmlvcml0aXplIGJ1aWxkZXIgcXVldWVz IG92ZXIgdGVzdGVyIHF1ZXVlcy4KKwogMjAxOS0xMS0wOCAgQWxleCBDaHJpc3RlbnNlbiAgPGFj aHJpc3RlbnNlbkB3ZWJraXQub3JnPgogCiAgICAgICAgIEJ1aWxkIGZpeCBmb3IgcGxhdGZvcm1z IHdpdGhvdXQgV2ViUlRDJ3MgU1NMIGFmdGVyIHIyNTIxODUKSW5kZXg6IFRvb2xzL0J1aWxkU2xh dmVTdXBwb3J0L2V3cy1idWlsZC9sb2FkQ29uZmlnLnB5Cj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xzL0J1 aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9sb2FkQ29uZmlnLnB5CShyZXZpc2lvbiAyNTIyMzkp CisrKyBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvbG9hZENvbmZpZy5weQkod29y a2luZyBjb3B5KQpAQCAtNzcsNiArNzcsNyBAQCBkZWYgbG9hZEJ1aWxkZXJDb25maWcoYywgaXNf dGVzdF9tb2RlX2VuCiAKICAgICAgICAgY1snYnVpbGRlcnMnXS5hcHBlbmQoYnVpbGRlcikKIAor ICAgIGNbJ3ByaW9yaXRpemVCdWlsZGVycyddID0gcHJpb3JpdGl6ZUJ1aWxkZXJzCiAgICAgY1sn c2NoZWR1bGVycyddID0gW10KICAgICBmb3Igc2NoZWR1bGVyIGluIGNvbmZpZ1snc2NoZWR1bGVy cyddOgogICAgICAgICBzY2hlZHVsZXJDbGFzc05hbWUgPSBzY2hlZHVsZXIucG9wKCd0eXBlJykK QEAgLTkxLDYgKzkyLDEyIEBAIGRlZiBsb2FkQnVpbGRlckNvbmZpZyhjLCBpc190ZXN0X21vZGVf ZW4KICAgICAgICAgY1snc2NoZWR1bGVycyddLmFwcGVuZChzY2hlZHVsZXJDbGFzcygqKnNjaGVk dWxlcikpCiAKIAorZGVmIHByaW9yaXRpemVCdWlsZGVycyhidWlsZG1hc3RlciwgYnVpbGRlcnMp OgorICAgICMgUHJpb3JpdGl6ZSBidWlsZGVyIHF1ZXVlcyBvdmVyIHRlc3RlciBxdWV1ZXMKKyAg ICBidWlsZGVycy5zb3J0KGtleT1sYW1iZGEgYjogJ2J1aWxkJyBpbiBiLm5hbWUubG93ZXIoKSwg cmV2ZXJzZT1UcnVlKQorICAgIHJldHVybiBidWlsZGVycworCisKIGRlZiBjaGVja1ZhbGlkV29y a2VyKHdvcmtlcik6CiAgICAgaWYgbm90IHdvcmtlcjoKICAgICAgICAgcmFpc2UgRXhjZXB0aW9u KCdXb3JrZXIgaXMgTm9uZSBvciBFbXB0eS4nKQo=